Forem: hong

OOP + S = OOPS

hong — Sat, 01 Jun 2019 01:37:45 +0000

OOP

When we first started with OOP it was about modelling objects after things. A more modern approach says that OOP is about the messages passed between these objects. But the more I draw message diagrams, the more I realize OOP is like a hidden object game. You are always discovering new objects that can be brought to life.

OOP Opponents

Some OOP haters, like Brian Will from Youtube, say that OOP makes you model objects that do not exist in real life. These are your 'Managers', 'Builders' and 'Coordinators". Yes these don't make sense in the real world, but what is the alternative? Long lines of confusing code? Documentation embedded into your code that lies? A spiders mess of modules with complicated api?

The only other viable solution to OOP is purely functional programming, but this comes with it's own set of issues.

I'll admit when I goto describe the big picture (class hierarchy), it can get blurry. The global scope is messy. However when I look at individual objects, the code is clear with little documentation needed. The local scope is clean. This is where UML diagrams for the big picture can help.

The S in SOLID

The S stands for Single Responsibility. A method, class or module should only be responsible for one thing. A method, class or module should only have one reason to change. So why is it so hard to implement?

S means Struggle

It's not just you. The single responsibility rule is completely arbitrary. It changes with the problem space. Newbies and seasoned coders alike still have problems. It get's easier with experience.

S means Spiral

If you follow the rule strictly, you might end up with classes with just one function. Then you will have too many objects with messy messaging between them. How do you know what should be another object and what should not?

It's not one answer fits all. I would say lean towards having more objects than less. Unless you can safely answer that this objects requirements will not change. If you find an object to be useless, you can always merge functionality.

However the opposite is not always true and may cause a spiral of changes.

S means Simple rule

Sandi Metz of POODR came up with a simple rule. Methods should be at most 4 lines long and classes should be at most 100. This is using the standard 80 character width. Try it out.

4 lines might seem too restrictive, but it's actually a lot. If you are having trouble, then you can split up your methods. If it's still too long, a hidden object might be waiting to be discovered.

OOP really means

OOP is about protecting your code against changing product requirements. The smaller your objects are the better you can defend. Adhering to the single responsibility rule helps with this.

OOP is about the 80/20 rule. 80% of the benefit should come from 20% effort. OOP can achieve this.

clean code

Writing clean code is not easy. I'm still refactoring my code and so should you. Go on a treasure hunt and find those hidden objects!

In defense of Test Driven Development

hong — Mon, 08 Apr 2019 19:31:17 +0000

In defense of Test Driven Development

I'm still learning to write great code like everyone else. Recently there's been some articles bashing TDD. I would like to share what worked for me with TDD.

OOP and TDD a good pair

Good OOP should follow SOLID and clean code principles. Classes should be small and methods even smaller. This makes writing unit tests much easier as there is much less to test per test case.

Clean code by Uncle Bob and POODR by Sandi Metz both dedicate large chapters to testing. This is not by accident. Test driven development is a huge part of writing maintainable code. If you are having problems with test driven development, perhaps your classes and methods are too long. Consider whether or not your code follows SOLID and clean code principles.

Test cases first

Before TDD, what I did was create a class and define the attributes and methods that I think I would need. I found this habit hard to break, until I forced myself to create test cases before I even wrote one line of code. How I did this was to create easier unit tests.

Python example:

def test_myobject:

    #check if the attribute exists
    self.assertTrue(hasattr(myobject, 'myattribute'))

    #check the type
    self.assertIsInstance(myobject.mylist, list)

    #check if function
    self.assertTrue(callable(myobject.mymethod)

    #check if attribute defaults to False
    self.assertFalse(myobject.my_false_attribute)

This is very similar to creating the basic structure of a class without writing the actual class. Some would argue that these unit tests do not test anything and are not worth it. After all why would you test basic language concepts like variable assignment?

I would say an object's public API is a contract and should be tested. Any change of these attributes will break code. This would be an easy fix, but also these unit tests are easy to write. This make it almost free.

Tips

Have your class and unit tests side by side when writing and updating code. If you update one, update the other.

If your constructors contain any conditional assignment logic, then you should definitely write unit tests for this.

I only write unit tests for methods that are public. If your private methods contain some tricky logic that needs to be tested, perhaps consider creating a mixin class for that method and exposing it publicly.

If your test cases import a bunch of other classes, that can show you are testing a too much. Practice DRY (Don't repeat yourself) in testing code as well. Use mocks and stubs in replace of other objects that your unit tests depend on instead of importing other objects.

Benefits

Good test cases have made my flow much better. Now I can rapidly design and add new features and they mostly work on first run. If I do have errors, the issue is with integration and not the single object itself. Refactoring is also easier as I know when I break stuff.

This is what worked for me. Good luck on your journey and help others along the way!

The skinny on Python dev environment and packaging (pip)

hong — Thu, 14 Mar 2019 18:48:01 +0000

Welcome to the wild world of Python dev. Where nothing is consistent and there is no such thing as as consensus.

Don't worry I'll be your guide.

Background

I quietly released pyquilted which generates nice looking resumes from Yaml in PDF. I learned a lot about setting up a python dev environment and packaging an app to distribute on Pypi (pip) and now I am sharing that knowledge.

You can find out more about quilted resumes here.

Unittests

Running test cases were for some reason harder than it should be. Follow these steps to make your life easier.

Create a directory called 'test' in your project root.

Name all your tests 'test_*'.

Put an __init__.py so that the directory get's recognized as a package.

running all tests on Python 2 and 3
python -m unittest discover

running one test
python -m unittest test.test_name

Venv vs VirtualEnv vs PipEnv

These are all ways to isolate your python pip environment so you do not pollute your main system python. Uninstalling a package and it's dependecies is a pain in pip. This avoids this by allowing you to easily delete and create new envs.

But which one do you go with?

Venv ~~vs VirtualEnv vs PipEnv~~

Just go with venv. It comes standard with python and is officially recommended post 3.3+. It also has less gotchas than virtualenv and I don't want to be gotten.

to create a new venv folder
python -m venv venv

to load the environment
source venv/bin/activate

turn off the venv
deactivate

Pipvenv is nice, but it's not standard and is based on virtualenv.

Python 2.7 and Venv

Venv doesn't support python 2.7, bummer.

Simple solution
pip install py2venv

This installs virutalenv for you and creates a wrapper around virtualenv. Meaning you can use the venv command. Great!

Where to put your venv folder?

If you are just running one venv per project, put it in the same directory as the project root and name the folder 'venv'. Make sure in your .gitignore you have added 'venv'—on github and gitlab this is added for you.

one venv environment per project
project_root/venv

The issues comes when you want different venv environments for different python versions you are testing. Here's a few options

multiple venv per project
project_root/venv/{{python_version}}

alternative, venv in home folder
~/venv/{{project_name}}_{{python_version}}

setup.py

Setup.py is like a makefile for your app. It's based off of setuptools. It normally comes pre-installed. If not you can install it like so

pip install setuptools

The documentation can be found here. Some parts are either confusing or not working as expected. I noted the parts here.

You can see an example of my setup.py in the pyquilted source

Command line entry point

You want to run your app in the terminal using your app's sweet name. This was a lot tricker than it should be. After endless searching, this method below worked for me.

In your project root, create a file named main.py. Have one function named main that calls your run function. An example here.

Add the following into your setup.py

entry_points={
    'console_scripts': ['command=pacakge.main:main']
}

Where 'command' is the command you want to use. 'package' is your package name.

For example my pyquilted entry point
'console_scripts': ['pyquilted=pyquilted.main:main']

Markdown readme

Pypi now supports markdown. Use a with open file handler to set your readme doc to a variable.

long_description=readme,
long_description_content_type='text/markdown'

Including non py files

There are two ways to add files to either your site_packages for data or your source package for test cases and such.

package data

Use the below format for adding data files to your site_packages.

package_data={
    'package_name': ['directory/*.pattern']
},
include_package_data=True

MANIFEST.in

This includes files in your source pacakge that are not in your site_packages folder. The format is a little weird.

include file   # include single file
recursive-include directory pattern # include a directory and files

requirements.txt

Skip it. It's not really necessary, but simple to make.

Enter Pyenv

Still crying over the disaster that was migrating from Python 2 to 3. Me too, join the club. We stil have to support Python 2 and test other versions of 3 as well. How to solve multiple python versions? We could learn Vagrant or Docker. Nah that's way overkill.

Solution Pyenv.

Install Pyenv by following instructions here pyenv.

list installable python versions
pyenv install --list

install version
pyenv install 2.7.15

list versions installed
pyenv versions

switch to python version
pyenv global 2.7.15

Debian, Ubuntu and others

On Debian, Ubuntu and other distros, you will get an error about ssl when trying to build older versions of python. The issue is that libssl 1.1 is not compatble with Python < 3.3. The solution is to download the dev headers for libssl 1.0 and link them manually when running pyenv install, per this thread.

CFLAGS="-I/path_to/libssl1.0-dev/include -I/path_to/libssl1.0-dev/include/x86_64-linux-gnu" \
LDFLAGS="-L/path_to/libssl1.0-dev/lib/x86_64-linux-gnu" pyenv install 2.7.15

Eggs everywhere

To stop python from creating eggs that you accidently upload to Pypi when you are testing.

Use the below command
pip install .

instead of this command
python setup.py install

Building and Wheels

Honestly wheel files are not necessary. The difference in install time between source and binary is like a few seconds. However it's really easy to build binary wheels so why not.

make sure you have wheel installed
pip install wheel

run this to build your packages
python setup.py sdist bdist_wheel

Finally upload to Pypi

Create an account on Pypi.org. Then install twine which is the official pypi app
pip install twine

Upload with this command
twine upload --skip-existing dist/*

The key is skip existing files, otherwise you get errors when uploading the dist folder with existing files.

Congrats!

Congrats you beat the final boss! Pat yourself on the back for finally deciphering python dev environment.

Truth about Template Engines Part 2

hong — Wed, 13 Feb 2019 19:49:52 +0000

My background

This is part 2 of my articles on template engines. Specifically today I am writing about the Mustache spec.

What qualifies me to pass judgement on templating engines? I have written web and email templates for fortune 500 companies. Some of which have been large gnarly order confirmations and airline reservations full of beautiful, un-readable business logic.

Enter Mustache

When logic-less Mustache came onto the scene. It did something amazing; which was to get other programmer's to embrace it and re-write it in their language of choice. Sandi Metz writer of POODR said that "if your design is good then others would want to copy it". I think this applies. Mustache was really onto something and logic-less really made sense.

Except Mustache is not logic-less. It contains conditional and looping logic. I would say it's a minimal logic templating language. You can also become full logic by embedding lambdas into your data is too. If you are thinking adding callable functions to your json data sounds like a bad idea, it is.

Mustache is not perfect. In fact if it were, it would be the most downloaded library for JavaScript templating. It's not, Handlebars is. Proving that logic is still king.

In the following, I will outline some parts of the spec that I think need some work.

What's in a symbol?

A quick rundown of Mustache's symbols and their definitions.

symbol	meaning
`{{ }}`	delimiters
#	begin section
/	end section
^	inverted sections
>	partials
&	unescape, safe-content
!	comment
.	current node operator?
<	proposed layout operator

Just in real life sometimes choosing the right words can go a long way. It's clear that the choices for symbols and their meaning were JavaScript influenced. However compared to other languages it may cause some confusion.

'#' in many scripting languages is a comment
'!' is negation in many languages
'.' it's not mentioned in the spec, but refers to the current variable?
'>' vs '<' partial vs layout will be confusing if implemented

Are these symbols any better than having the actual word? To any non-programmer at first glance these symbols make reading templates worse. Compare to this:


{{if section }}

{{not section }}

{{each item }}

Truthy or Falsey

One of the tenants of Mustache is that it aims to be language agnostic. This is at a cost of clarity between language implementations.

The below Mustache template will render differently depending on which language it's rendered on.

Template:

Shown.
{{#person}}
  Never shown!
{{/person}}


Hash:

{
    "person": false  // or '', 0, "0", [], {}
}

Below is a brief table of some of the different Truthy evaluations in scripting languages.

lang	''	0	"0"	{}
Php	False	False	False	True
JavaScript	False	False	True	True
Python	False	False	True	False
Ruby	True	True	True	True

This variability in the spec is in my opinion one of it's weaknesses. From a business perspective, it doesn't make any sense why this exists other than to make it easier for developers to implement the spec. Templating engine developers should not be the target audience for a template language.

I know that others have voiced their opinions on this matter and the community seems divided. However consider this conversation. "If we migrate to a new backend, our clients might see bugs in their templates because of the way Mustache is designed". Try selling that to your managers.

This might seem like a non-issue, as you are not likely to transfer your templates between languages. However when you are choosing a templating engine to be used by non-programmers, stuff like this can bite you. Also backend re-writes do happen.

Almost any standardization would be better than no standardization at all. It's just the idea that tying evaluation of of a template block to a language's own internal truth table makes no sense. After all it could be the case that I want to show that 0 items are available.

In my professional experience, it's almost always safer to be explicit when rendering a conditional block. There is no equivalent problem in other templating languages because you can always just do an '==' comparison. This is explicit.

The problem arises because Mustache aims to be logic-less which is a mismatch for some template blocks use cases. Going with Python's model of everything Falsey is good enough. Just please not PHP.

Whitespace

Whitespace is sacred. One of my biggest pet peeves with templating languages is that they always clobber whitespace. Whitespace is ambiguous. The Mustache spec is also loose with regards to whitespace. Consider the below Mustache code:

Is this a variable or section?

{{ #person }}

Some developers have solved this ambiguity by requiring that mustaches be continuous e.g. {{#person}}. However consider the below:

What's easier to read?

{{name}} vs {{ name }}

{{#person}} vs {{# person }}

In a perfect world, whitespace in general should be preserved. This is difficulty to implement as whitespace is ambiguous by nature.

Should this {{variable}} preserve spacing if it is null? Is it one or two spaces?

How about pre, post and nested newlines in a null section? Indents too? It gets complex.

         {{#mysection}}
              {{nested}}
         {{/mysection}}

The spec is unclear on all these scenarios. Some implementations have restricted spacing while others have not. It is the same with preserving whitespacing. This result is too much variability in implementations. This could be cleared up with a more clear spec.

Template Inheritance

One of the most requested features is template inheritance. This is when you have a base layout that other templates can inherit. For example a layout with a header and footer and other templates can then render the body content; optionally they can override parts of the parent layout.

My answer to this is No. Just No. This is a nice to have and perhaps should be an optional spec. However Mustache does not need this. I don't think this flows to with the concept of logic-less. Partials are already good enough. They are themselves a bit too much logic.

Wishlist for Mustache 2.0

What's better than logic-less? More logic-less! My wishlist for Mustache 2.0 involves implementing NO new features.

remove lambdas feature
remove set delimter feature
standardize truthiness
standardize whitespace rules
No template inheritance

Mustache 2.0 done and done

Don't get me a wrong. I'm a huge fan of mustache. However like writing code, we can refactor to make it better.

Firefox vs Chrome, MD two space vs GFM line breaks

hong — Wed, 16 Jan 2019 14:27:06 +0000

Hi guys,

I submitted my first post yesterday. My markdown that I pasted in had line breaks because it makes it easier to adhere to the 80 char rule while looking nice in editor(vim). I use traditional 2 space rule for line breaks.

I submitted my post and it looked fine on Chrome and ugly in Firefox. The content had extra line breaks everywhere in FF. It looks like Firefox supports GFM style linebreaks while Chrome does not? Is that for real or am I missing something?

Thanks,

Truth About Template Engines

hong — Tue, 15 Jan 2019 16:42:33 +0000

Intro

I'm writing about content template engines today like Shopify's liquid,
Python's Jinja, Mustache and the others. I am here to finally answer these
burning questions I'm sure you all have.

What's better Logic-less vs Logic-full?
Why are there so many?
How to choose one?

What qualifies me to pass judgement on templating engines? I have written web and email templates for some of the biggest fortune 500 companies. Some of which have been large gnarly order confirmations and airline reservations full of glorious, unreadable business logic.

This is part 1 of a 2 part series.

Let's define logic

Logic-less templates became popular with the arrival of Mustache. Except
Mustache is not really logic-less. There's control flow, looping and even
custom code via lambdas. If you use Mustache without lambdas, I would consider this as close to logic-less as you can get while still being useful.

Full logic in my opinion is when you are freely able to mix both content and
any arbitrary code into your templates. Examples of such are ERB, XSLT and
Underscore.

Some where in-between are semi-logic templates. Logic can be introduced via pre-defined filters and/or custom filters. Semi-logic templates, if abused, can lead to full logic templates. This category includes Liquid, Jinja2, Handlebars, Nunjucks and many others.

'View as an app' frameworks like React and Vue, I would consider to be full logic templating engines plus everything including the kitchen sink.

Logic-less vs Full Logic

In a perfect world your data lines up exactly how you want it to look in your presentation layer. You never need to write any business logic in your templates. This is where logic-less templates is a perfect match and what everyone should strive for.

These style templates are very easy to read. Anyone can understand and write these. Contrast this with full logic that are hard to read and hard to write.

So why use full logic? There are advantages to full logic templates. They are extremely flexible and you can hack quick fixes when you don't want to write complicated controller logic. I've written some crazy business logic into templates because there are always those clients whose requirements are never sane.

In my experience with working with many clients, the data almost never lines up with the presentation requirements.

Know your audience

Not everyone is a programmer. From my experience, anyone trying to teach anything beyond simple control structures to a large group of non-programmers, is going to have a bad time. Having your template engine be simpler gives it access to a broader range of technical expertise.

Imagine if Shopify had used ERB. This would have been a headache to train all customers and have random scripts to solve for business logic. Template writers are not backend coders. This would have also have been a big risk given anyone access to freely execute ruby code through templates.

Even for large teams of experienced programmers, I would still say it's an anti-pattern to use a full logic templating engine. Full logic just leads to HTML content mixed with ugly code. The logic embedded in the code will be hard to test and will introduce bugs.

I wrote XSLT in a large team of experienced technologists. XSLT is in itself a verbose and complex language for templates. Simple changes were fine, but anything greater required hours to write and test. Yet we still did it, largely to solve for gaps in our product.

Is it ever okay to go full logic?

Yes, I would say under certain circumstances, it's okay to use full logic templates. If your team is small and experienced developers you will be fine. However if you find yourself leaning on it too much, it could mean there are some problems with your design. You will start building technical debt when your team becomes larger and less experienced.

Why are there so many?

They are fairly straightforward to write. It seems like each language has their own versions of full logic and semi. Python and JavaScript seem to be leading the pack with more still being created.

Another reason is the 'Not Invented Here Syndrome'. Shopify's Liquid and Mozilla's Nunjucks might be guilty of this. Although it is probably more for legal reasons.

Template engines evolve. For example Twitter liked Mustache, but added template inheritance and wrote Hogan. Ember's Handlebars also built off of Mustache by adding an AST to make sure generated templates are html correct. This is nothing new in the open source world.

Overall I think the number of template engines is good. A template engine can be very personal. Repeating what I mentioned above, what one person needs in a template engine another may not. Perhaps programmer's just want to scratch an itch.

How to choose the one?

You always want to be further away from logic-full as you can. If you can get by with just Mustache and not use lambdas; you should do it. Otherwise Logic-less with some filters/helpers like Liquid and Handlebars is probably the sweet spot.

But please never go full logic.

Which one do I like best? None. The current crop of template engines are all adequate, but not perfect. Certainly room for another templating engine.