Forem: Daniel Brady

Guarding Makefile targets

Daniel Brady — Thu, 23 Jul 2020 03:44:58 +0000

This may just be my favorite one liner of my own devising in recent years.

I love it because of a combination of its utility (🔧), the story behind how it works (💭), and its power to induce headaches to its readers (😅).

It belongs in a Makefile :

## Returns true if the stem is a non-empty environment variable, or else raises an error.
guard-%:
  @#$(or ${$*}, $(error $* is not set))

And has a simple function:

➜  make guard-FOO
Makefile:12: *** FOO is not set.  Stop.
➜  FOO=42 make guard-FOO
➜

Deciphering from top to bottom, left to right:

The % symbol used in the target declaration a “static pattern wildcard”, and in more familiar regex terms it translates to guard-(.*) ; the “stem” of the pattern, i.e. the value of the match group, is stored in an automatic variable called $*.
The @ prefix prevents make from printing out the command it is about to evaluate. (Hint: Take this away to actually see what’s happening.)
The # is your typical shell comment character: anything that comes after it will be interpreted by the shell (and make) as a comment, and thus ignored in a "truthy" way.
The $(or …, …, ...) is a make short-circuiting conditional function: it expands each item until it finds one that expands to a non-empty string, at which point it stops processing and returns the expansion. (Note: It does not evaluate it.)
The ${…} notation is the make syntax for evaluating a variable.
The $* as mentioned previously, holds the value of the matched pattern in the target name.
The $(error ...) is a make control-flow form, and raises an error as soon as it is evaluated, after allowing for its arguments to be expanded.

So, putting it all together: FOO=42 make guard-FOO expands to:

#$(or ${FOO}, $(error FOO is not set))

which in turn expands to:

#$(or 42, $(error FOO is not set))

which, finally, expands to:

#42

which is evaluated by the shell as a comment and ignored, but in a truthy way.

However, make guard-FOO expands to:

#$(or ${FOO}, $(error FOO is not set))

and then:

#$(or , $(error FOO is not set))

and then:

#$(error FOO is not set)

and then a fatal error.

The value of this is as a prerequisite for other targets, to easily guard against missing environment variables, e.g.

say-something: guard-SOMETHING
  echo ${SOMETHING}

➜ make say-something
Makefile:12: *** SOMETHING is not set.  Stop.
➜ SOMETHING='Hello, world!' make say-something
echo Hello, world!
Hello, world!

Have you ever tried implementing a language spec?

Daniel Brady — Mon, 24 Feb 2020 10:34:00 +0000

Recently I was reading through Safia(@captainsafia)'s post, where they dive into the JavaScript spec to discuss the implementation of the Array.prototype.filter function:

The fun of filter, according to the ECMAScript spec

Safia Abdalla ・ Jan 22 '20

#javascript #spec #filter #array

Reading through the algorithm, I felt the urge to implement it. I guess I'm just programmed that way 😆

I didn't, though. I've thought many times that it would be a fun challenge to implement a language spec, and a great way to learn about language design and the choices made by the creators of my tools. But I still have never tried.

If you've tried, what was your experience like?
If you haven't tried, why does or doesn't it sound like fun? 🤔

(I'm asking specifically about implementing an existing language spec; creating your own sounds fun, too, but not the topic I'm introducing for discussion in this post 😄 another time, perhaps!)

syndicate: Harvesting Commits and Drafting to DEV

Daniel Brady — Sun, 23 Feb 2020 19:44:00 +0000

I made this for people who write words they share with others:

dabrady / syndicate

a simple implementation of the POSSE content publishing model

It distributes copies of the content you create to various publishing sites automatically.

It took me a week of focused evenings and a weekend of dedicated hacking to polish it to my liking; and another week to write about it.

This is how I did it.

With the skeleton of a project in place and a clear direction to go in, I was ready to get started in earnest.

I decided to get the DEV.to API interactions working first, and then I would need to figure out how to harvest a git commit for the actual input to DEV.

Creating basic wrappers for the "create" and "update" end points of the DEV.to API went fairly smoothly and quickly: I'd already front-loaded the learning curve earlier with my "fetch" implementation.

Now came the bits which would prove to give me the most challenge in this project: doing things with git. This is where the Github Workflow and Action documentation became truly valuable: it showed me that I had access to many git-related things concerning the commit that triggered the workflow, exposed to my action in the form of environment variables.

For my purposes, the most valuable of these would turn out to be:

the GITHUB_TOKEN (used for authenticating requests to the Github API)
the GITHUB_SHA (the commit SHA that triggered the workflow)
the GITHUB_REPOSITORY (the full name of the repo containing the workflow)

It quickly became apparent that the actual contents of the commit which triggered the workflow were not available to me out of the box; I would need to get them myself, somehow. I had started this project expecting the only external APIs I'd be interacting with would be the ones for the silo platfroms I included support for; but this new development meant I needed to learn about the Github API, as well.

If I had opted to use the Github API directly through the requests package, like I was doing with the DEV.to API, I would have had a very different experience (though not necessarily a better or worse one). But I did not do that.

Instead, I went looking for a Github-specific Python library to use. I don't remember why I chose this path, but it might have simply been because I knew so many Github extensions and integrations existed that there were bound to be some who had wrapped the Github API into an easier-to-consume Python library.

And indeed there were several options to choose from. The first option advertised by the internet via Google was PyGithub. But the documentation felt terrible: a full half of the README was simply bragging about who was using the library and linking to people on the internet recommending it. For that reason alone, I almost immediately went searching for some other option.

The package I decided to use was called github3.py. By comparison, the documentation felt wonderful, like the package authors actually cared about helping people understand the software, and not just each bit of code in isolation. (NOTE TO SELF: don't be a hypocrite, create a great ReadTheDocs site for syndicate).

But there was a problem: adding github3.py to my project dependencies broke everything.

The root cause, again, was that my Docker image did not create the environment necessary to support the code I was trying to run. I quickly discovered that switching from using python:3-alpine back to python:3 resolved the build issues, though it also brought back the "this thing takes too damn long to build" problem. But I was impatient, so I decided to put up with the longer build times to unblock what felt like the "real project development" and deferred solving this performance problem until the end of the project.

🐌🌸 Note From Future Me
I would eventually switch from using github3.py to using PyGithub (which did not introduce the same build issues), and if I had just tried it out at this point instead of being so impatient to continue feature development, I could have saved myself a lot of time and self-induced frustration felt while waiting for my code to compile.

With the build back to a stable state, I finally began to tackle the challenge of figuring out how to transform a git commit into a draft on DEV.to.

The github3.py API documentation is great, and I quickly figured out how to extract the contents of the added/changed files in the commit that had triggered the action. I realized, however, that I couldn't assume all the changed files in the commit were posts that needed syndication; I needed a way to identify the files the commiter intended to syndicate.

A quick and easy solution was to introduce a configuration variable for my Github action specifying a path prefix that could be used to distinguish 'posts' from other files in their repository. This has downsides: it assumes the author keeps all their posts in a particular location, and it assumes that all files in that particular location are posts.

That said, it would suffice until it actually caused someone problems, so I went with that approach. I opted to make it an environment variable instead of an explicit input to the action because of how Github Workflows work: this value is static information about the repository, and my action might get used in multiple steps within the same workflow; using an environment variable allows it to be specified at a higher level than the individual action config if you want, which let's you share the value across a larger part of your workflow and avoid duplication.

Now that I could accurately identify "posts" in a given commit, I needed to be able to parse out the details necessary for the API call that would be made.

To create a post, the only actual data required by the DEV.to API is a title. Simple. Now that I could access everything about the files being syndicated, I could just grab their titles and contents, plug them into the API request, and win at life.

But...wait. Where is the title of a post? 🤔

An obvious, but probably poor, answer is "in the file name." I could already tell that going down that route would lead to pain and suffering, so I decided to make another assumption: my action would expect all posts to be written in Markdown, and contain a YAML frontmatter with at least a title attribute.

Given my initial target audience are those who write for DEV.to, and that is how posts are formatted there, it isn't as random of a choice as it might first appear. But still, I did not do any research on the prevalence of this format adoption, and that may come back to bite me in the future.

But for the first iteration, it would do just fine. I now needed a way to parse YAML out of a string; I knew YAML is pretty widely used in software configuration, and Python is known for its usage in data processing, so I expected there to be a good library for working with it.

And indeed there was. PyYAML seemed canonical, from what I could tell, but it also seemed like overkill for my needs; plus, the documentation was ugly and hard to use (again NOTE TO SELF: ugly docs discourage users). I kept searching, and in one of my queries I included the keyword 'frontmatter' and caught a lucky break: python-frontmatter was a small library specifically for manipulating YAML metadata embedded within text documents, and perfect for my needs.

At that point, it only took a bit of tinkering and suddenly I had something that would automatically push my newly created content to DEV.to as a draft, when I pushed it to my repo. It was time for my happy dance.

But the game had only just begun: I now needed to determine if I'd previously created a draft for a given post in order to avoid creating duplicates, and set the stage for pushing updates to existing content if possible.

syndicate: Prototype, Requirements and Design

Daniel Brady — Sat, 22 Feb 2020 15:01:00 +0000

I made this for people who write words they share with others:

dabrady / syndicate

a simple implementation of the POSSE content publishing model

It distributes copies of the content you create to various publishing sites automatically.

It took me a week of focused evenings and a weekend of dedicated hacking to polish it to my liking; and another week to write about it.

This is how I got started.

After my initial research and setup, I was eager to start spec'ing out the actual thing I intended to build. I began by converting my implementation of the "Hello, world!" Docker action tutorial into a Python project that would run in a Docker container.

The first challenge I overcame was finding a Docker image that would actually run Python. As it turns out, the alpine image used in the tutorial does not come with Python installed, so I needed to either install it myself or look for a different container that came with it. I opted for the second approach first, and a simple search of "python Docker image" led me to...the python Docker image.

At first, I simply swapped out my alpine image for the python:3 image, but that significantly increased the action build time (by 2 or 3x 😬) so I was immediately skeptical of using it. Luckily, I decided to read beyond the "How to use this image" section of the documentation, and discovered there were a couple of 'light-weight' versions. Since I didn't yet have any project dependencies, I decided to try using python:3-alpine because of its boasted performance characterisics, and then forget about it until it no longer worked. It did the thing, and my base build time was only about 6 seconds.

Now that I had a Github Action building a container that could run Python, I needed to figure out how to use the DEV.to API I had discovered earlier. But more importantly, I needed to figure out how to do it in Python, ideally using some sort of standard or canonical library.

The requests package was the answer, quickly found, and its documentation was to become the third and final sacred text in my project's development.

I started to familiarize myself with requests and the DEV.to API by transforming my existing action code into a simple prototype. Given my inexperience in basically everything about this project, I decided to start out building something that would react to pushing a commit to my Github repository by fetching my posts from DEV.to and logging out their titles. Reading from DEV was the simplest interaction with the API that seemed like it would also make for a good exercise.

Once I got that working, I thought about whether I would actually need that functionality in syndicate. It wasn't clear at that point, because I hadn't actually done any design work for the project yet.

I decided to spend time maturing this first bit of logic into an example for myself to follow later on, with respect to using the DEV API and requests package properly. I would include things like request authentication, input validation, error and failed request handling, testing, and logging.

This decision proved quite valuable. It gave me a small but wholistic exercise in which I figured out how my overall application logic might flow, and learned things like how to print to the Github Workflow log; how to read information from the Github Workflow environment; and how to use Pytest to write specifications for my code.

By the end of that exercise, I had:

a Github Workflow definition that used my action and provided it with a list of platforms to publsh to, along with the API keys required for using their APIs
an entrypoint for the Docker container that invoked my 'main' Python application with any arguments specified by the Github Workflow
a small but functional Python module called 'syndicate' which defined a single function, elsewhere, responsible for basic input validation, delegation to the appropriate syndication code, and final reporting
a function that fetched and returned my published posts from DEV.to using the API key provided by the Github Workflow
the beginnings of a utility library, with functions that handled printing to the Github Workflow log at various log levels
a small test suite for my DEV 'fetch' logic
a fully polished and tested example of how to interact with the DEV.to API via the requests package

And let's not overlook the experience gained using Python itself!

I was happy with my prototype, and I had learned many useful things. At this point, I paused development to take stock of where I was and where I needed to go.

I spent an evening working on a set of initial requirements and high-level technical design for the actual thing I wanted to build, and this is what I came up with:

Initial Requirements

My MVP (and possibly first release) would only support DEV.to
It needed to be easy to add support for publishing to new platforms
Given a Markdown file, it needed to be able to create a draft on DEV.to with its contents
Given a (presumably changed) Markdown file previously syndicated to DEV.to, it needed to be able to replace the syndicated copy with the new version
Given a commit that triggered a workflow using the syndicate action, it needed to be able to identify and extract the contents of the added/modified files to syndicate from that commit
It needed to be able to identify files that had been syndicated to a given silo previously, in order to decide whether to create a draft or push an update

High-level Tech Design

syndicate would be designed as a Github Action, to be incorporated into a Github Workflow for managing content syndication
Inputs:
- a list of 'silo' platforms to syndicate to
- any secrets required to use the APIs associated with the given silos
Outputs:
- a summary of the files added/modified on each silo platform
Mechanisms for interacting with a particular silo should be encapsulated in their own 'adapter' files with a single point of entry; the syndicate.elsewhere function would be responsible for engaging the appropriate adapters and aggregating the output
Syndicated files would be 'tagged' with unique identifiers provided (presumably) by the silos themselves in the 'draft' API response; this tagging would allow the action to recognize files in the repo that already exist on a given silo

Seeing it laid out on paper (yes, that still exists) was getting me excited! I gutted the prototype I had created, keeping only the main structure and wiring along with some of the more useful code snippets as comments, and set to work.

syndicate: Initial Research and Setup

Daniel Brady — Sat, 22 Feb 2020 14:56:00 +0000

I made this for people who write words they share with others:

dabrady / syndicate

a simple implementation of the POSSE content publishing model

It distributes copies of the content you create to various publishing sites automatically.

It took me a week of focused evenings and a weekend of dedicated hacking to polish it to my liking; and another week to write about it.

This is how I got started.

In essence, the thing I wanted to build would take changes I made to files in a git repo, and push them someplace else. Classic Patrick Star.

One of my first questions was, "Can I communicate with DEV.to programmatically?" DEV.to is currently the only place I publish writing, so if the only answer involved pretending to be a human and programmatically engaging with the UI to manipulate blog posts, I was likely to abandon the idea (though I probably would have tried it just for fun).

Thankfully, my project was saved by a quick Google search: DEV has a beta API that exposes end points I could leverage.

My next question was one of automation: how to trigger a DEV.to API call in response to changes in my git repository? My immediate reaction was "git hooks."

I've played with git hooks in the past for doing things like automatically injecting a JIRA ticket number into my commit messages when pushing; and automatically running database migrations locally after pulling if the schema changed or migration files were added. They seemed like the perfect mechanism to use for this project.

Recently, though, I noticed something called Actions show up on my Github repositories. I didn't really know what they were, but on the surface it seemed like a Github service implemented on top of git hooks.

I hadn't really imagined that the tool I would build might have any actual interface besides an automatic trigger. Thinking about Github Actions made me realize it would be nice to have my tool respond to changes on a remote repository (i.e. on a push command) and run on someone else's computer, rather than operating on a user's local machine; it would minimize "works on my machine ¯\_(ツ)_/¯" headaches when installing and using the tool itself. And it would be awesome to have some sort of interface for monitoring progress and output, and maybe even a logging mechanism.

Github is my private choice of remote source control platform, so I looked into Actions and how to create them. I was pleasantly surprised at the extent of Github's documentation on the subject, and it would become the first of a few sacred texts I relied on throughout my project.

The docs had two tutorials: one using JavaScript and one using Docker. I chose to follow the Docker tutorial because I'm not familiar with container technology and it could potentially benefit me in my upcoming transition to a DevOps role at Tapjoy.

I dedicated a few evenings to the "Hello, world!" tutorial, trying to massage the steps into something close to what I imagined I'd need for my real project. I took my time, and gave particular focus to the parts concerning Docker: I wasn't sure this would be a proper fit for my project, so I wanted to be sure before I dismissed it as an option.

I'm glad I went slowly at the beginning. By the time I had completed the tutorial to my satisfaction, I had learned two important things: a Github Action would be a good way to manifest this tool, and I didn't want to build it out of shell scripts.

I can count on one hand the number of devs I know who can effectively read shell scripting languages, let alone effectively write them; I myself only have a passing proficiency at shell scripting, and I wanted this project to be easy for others to grow to suit their needs.

Python was a choice I was primed to make: it's a language I had virtually zero experience developing with; it's easy to pick up and is widely used; it's been on my list of tools to familiarize myself with for awhile; and I have a friend who is enamored by it and is always touting its benefits in web programming.

I'd used Python before, nigh on a decade ago. But if I wanted to start this project off in a good direction, I needed to brush up on contemporary best practices. My primary concerns, at least in the beginning, were project structure and development tools. How do I lay out this project, and what will help me build it?

The internet doth provide: The Hitchhiker's Guide to Python was often referenced, so I concluded it must espouse a highly regarded opinion and decided to adopt it as another of my sacred texts. Indeed, it proved very useful as a quickish reference for many things: project structure, package management, testing tools, documentation practices....

Armed with an initial (and empty) project layout and a basic Github Workflow that would execute my "Hello, world!" Action any time I pushed to Github, I could finally get started on the logic that would become syndicate.

syndicate: Why I Made It

Daniel Brady — Tue, 18 Feb 2020 21:01:00 +0000

I made this for people who write words they share with others:

dabrady / syndicate

a simple implementation of the POSSE content publishing model

It distributes copies of the content you create to various publishing sites automatically.

It took me a week of focused evenings and a weekend of dedicated hacking to polish it to my liking; and another week to write about it.

This is the first in a series of writeups on why I did it, how I did it, and what comes next.

Why's are important to me. I don't like doing things with my time without a reason, even if that reason is as vague as "huh, sounds like fun." Why's help me figure out if I care enough to continue, by situating my actions into some wider context. I've learned that most of the time, when I find my interest in something is waning, it's because I've either lost site of my why or because my why is no longer good enough to keep me coming back.

In the case of syndicate, my why started from one of those complex moments of understanding that often follow deep dives into rabbit holes, where a lot of things you've been processing in the background of your mind come suddenly to the front.

On this particular occasion, I was researching the subject of blogging. I had only recently written my first blog post, and so I had a lot to learn. I read about many things: the origins of blogging in the "weblogs" of the late 1990s; various techniques I could use to help me during the writing process; the communities fostered by different blogging platforms. It was this last one that eventually lead me to the IndieWeb and the concept of POSSE.

POSSE is an abbreviation for Publish (on your) Own Site, Syndicate Elsewhere, a content publishing model that starts with posting content on your own domain first, then syndicating out copies to 3rd party services with permashortlinks back to the original on your site.

This makes a lot of intuitive sense to me, since it is basically the same process used in the creation of more traditional forms of media like books and magazines: the author distributes copies of their work to the publishing companies they want to work with.

But more importantly, it reminded me that I'm not in control of the internet, and if I use the internet to create content, instead of creating content for the internet, I might not be in control of that, either. I don't care so much about possessing what I create, so much as I care about having some measure of control of and responsibility for it.

I decided I would adopt POSSE, and started thinking about how I would do it.

I felt this approach to content creation could be boiled down to a few simple steps:

Create a "source of truth" for your writing.
Know where you want to distribute, and how to do it.
When your truths change, ensure they are ferried to the right places.

It didn't take much imagination to see the parallels with software deployment. I could easily picture how it would work:

Create a git repo for my writing.
Decide on the platforms I want to distribute to, and write scripts to do so via public APIs.
Use git hooks or some other mechanism to run those scripts when changes are made.

I was immediately taken with this idea. Automating my jobs away is a minor passion of mine, and when I'm the only stakeholder, I like to indulge in a little premature automation just for the fun of it. I mean, who doesn't like makin' magic?

And it looked like a fun challenge! There was a clear objective and definition of success, and it seemed like something that other people besides myself would find useful.

So I made a Trello board, created a repository on Github, and started Googling.

I don't prep for the 'tech' part of tech interviews; does that work for you, too?

Daniel Brady — Tue, 18 Feb 2020 11:31:00 +0000

This has always been met with vehement opposition, feelings of being offended, and made sages out of friends as they become suddenly protective of me and wanting to "steer me right."

So let me just preface this with, "This works for me, I'm open to improvements, and I'm not disparaging what works for anyone else."

I don't prep for the 'tech' part of tech interviews.¯\_(ツ)_/¯

It has nothing to do with having a large ego or confidence in interviewing or anything like that, but rather more to do with how I judge myself.

It's mostly an extension of my approach to any sort of examination or evaluation growing up: if the progression of my life thus far hasn't already prepared me, then I'm obviously not ready for whatever the next step is, regardless of what the examiner says. Intentional preparation for things like exams has always felt...fake, I guess? If I pass an evaluation after spending days or weeks of studying for it specifically, then what is the evaluation really telling me about how well I'm prepared for whatever comes next? Isn't it mostly just telling me how well I passed the test?

When I transitioned out of a formal school setting and into "the real world" several years ago, that attitude towards evaluating myself didn't change. Job interviews are very obvious forms of evaluation, and I approach them with the same attitude: if I'm not ready, I'm not ready, and stressing myself out to get a job I'm not ready for doesn't help anyone, especially me. Failing an interview helps me learn where I should improve if I want a similar role in the future.

It's worked well for me so far: I've held two professional software jobs, and I started both of them with minimal self-doubt and virtually no imposter syndrome.

That said, I've had a few interview experiences that have revealed what I consider a major flaw in my approach: it assumes too much about the ability of the examination process to adequately and accurately evaluate.

Basically, as a student I tended to think that if I did not pass an exam, I was not ready to proceed in the coursework. Similarly, I now tend to think that if I was not given an offer after an interview, then I was not a good fit for the role.

But the reality is that accurate evaluation of people is hard; any feedback (external or internal) about yourself and your abilities has to be taken with a healthy amount of skepticism.

And this is why I specifically said I don't prep for the tech part of interviews. The tech part is the simpler (but not necessarily easier) part for me: once I'm asked a question, my own reaction to it can generally tell me how equipped I am to approach a solution to it, and hopefully that tells me something about how equipped I am for the role I'm trying for.

The real challenge of interviews, for me, has always been the part where I'm supposed be interviewing them as a prospective employer and potential teammate. 😓 This is the part of tech interviews I often forgot about early on and actually try to prepare for these days, because my success in that area relies on my ability in judging others accurately, and as I've tried to establish previously, that's hard.

Anyways, this is getting longer than I intended. I mostly wanted to share my flawed-but-works-well-for-me approach to tech interviews, and I'm curious if anyone else follows a similar approach. If you follow a different one, why does it work for you more than others you've tried?

syndicate: a POSSE way to DEV

Daniel Brady — Thu, 13 Feb 2020 11:26:00 +0000

Until a month ago I had never written a blog post.

My recent discovery of DEV has a lot to do with why I started, and indeed, I created my first post right here on DEV.

While educating myself on the art of writing for other people, I came across the term "POSSE", which I had never heard of before but is apparently kind of a big deal. It is a principle that is embraced by DEV itself, according to Ben.

Ben Halpern • Jul 21 '19

Hey, this is really on point, and we’ve been working on a lot of features to help people pull off POSSE in a simpler manner, while also trying to roadmap a future where trust is pretty clearly baked into our product.

Medium Was Never Meant to Be a Part of the Developer Ecosystem

Ben Halpern for The DEV Team ・ Jun 3 '19

#meta

Project Benatar: Fending Off Data Black Holes

Ben Halpern for The DEV Team ・ Jun 17 '19

#meta #opensource #projectbenatar

I’m not sure if you’ve read those, and some of the other posts we’ve made on the subject, but let me know what you think!

The idea intrigued me, as do most things that involve "write once and reuse" concepts. I began to imagine how such a thing might be accomplished, and just like P.O.S.S.E. itself, the recipe for its implementation seemed simple on the surface:

Create a "source of truth" for your writing.
Know where you want to distribute, and how to do it.
When your truths change, ensure a process is triggered to ferry your changes to the right places.

I don't know about you, but that sequence of steps strongly reminds me of the basic recipe for deploying software. And so I immediately re-imagined it as such:

Create a git repo for my writing.
Decide on the platforms I want to distribute to, and write scripts to do so via public APIs.
Setup git hooks to run those scripts when changes are made.

Long story short (the "long story long" will come as a separate post), I took a week of evenings and a weekend, and made this:

dabrady / syndicate

a simple implementation of the POSSE content publishing model

syndicate

A simple implementation of the POSSE content publishing model, packaged as a Github Action.

Write your content, store it on Github, and use this action in a workflow to draft it to silo platforms like DEV.to. The action will keep the silos up to date with your latest changes here on Github.

Wherever possible, when content is syndicated to a silo for the first time, it is created in an unpublished/"draft" form. Any exceptions to this will be called out in the documentation for silos below.

The silos currently supported are:

https://dev.to

A key assumption made by this project is that your content is written as Markdown, and includes a YAML frontmatter containing at least a title property. If this is not the case for you, this action sadly does not currently support your worflow. If you would like to see support for some other way of making…

View on GitHub

As the README describes, it will let you write your content on Github, and distribute it automatically to DEV.to, keeping subsequent changes in the content on Github synced to DEV.

It is designed to be used as an action in a Github Workflow, but technically it is just a simple Python program that can be run with Docker if you want (I include a Dockerfile). To use it as intended, all you need to have is a Github repository, and all you need to do is create a Github workflow in that repository which uses dabrady/syndicate in one of its steps.

For now, it only recognizes one silo platform, DEV.to, because that's the only platform I currently publish on. But I made it simple to add plugins/adapters for other silo platforms, so if you'd like to contribute, that's a great way to do so!

Like my first blog post, this is my first project written with the intent to share with other people. It's also my first time using Python in almost a decade, so I used The Hitchhiker's Guide to Python as my main resource for project management and layout. Any and all feedback is greatly appreciated! 🙏 I hope it benefits us.

What's your current favorite shell command?

Daniel Brady — Fri, 31 Jan 2020 12:46:02 +0000

What's your favorite line of code?

Daniel Brady — Sun, 26 Jan 2020 23:22:35 +0000

Abstract answers and actual code snippets are both welcome 😄

Iterating with loops

Daniel Brady — Sun, 26 Jan 2020 20:31:54 +0000

Cover photo by Sebastian Alvarez on Unsplash

Previously, on "Iteration"...

Iterating with recursion

Daniel Brady ・ Jan 25 '20

#beginners #thinkdeep #programming

I introduced the concept of iteration by way of a whimsical allegory about counting a staircase. To count the steps of a staircase, we first came up with a practical definition of what the "length" of a staircase is:

The length of a staircase can be described as the value of the first stair, plus the length of the rest of the staircase.

We then used that definition to derive an intuitive algorithm for calculating staircase length:

When counting the steps of a staircase, if you're at the top, stop counting.
Otherwise, add 1 to the result of counting the rest of the staircase.

My implementation of that algorithm showcased the elegance of recursion, which deconstructs a problem into a composition of the solutions to its parts. And we saw that recursion lends itself well to wholistic computation, i.e. situations in which you only care about the final result.

Constructive iteration

Today, I want to talk about other ways of approaching iteration that might lend themselves better to constructive computation. I'm using the term constructive in the mathematical sense, which I just learned about recently and think fits this topic particularly well.

A constructive process, in my mind, is one that gives you insight into the progress of the computation it is performing.

To keep with the "staircase traversal" imagery from my previous discussion of iteration, I imagine a constructive traversal of a staircase to be one in which, after each step, you know exactly how many you've climbed.

It's like you're actually building the staircase as you go, keeping track of how many steps you've added so far. Once you've decided to stop, you already know exactly how many steps are in the staircase without further computation.

We-Who-Code often refer to this type of iteration as looping, which captures the idea that each step you take closes a 'loop' in a larger 'circuit' of computation.

When looping, you can stop at the end of any step without worrying so much about unfinished work: if you stop precisely at the "end" of your computation, your record-keeping will reflect the solution to the whole thing; on the other hand, if you stop early, you'll still have something to show for it, it might just be an approximation of the result you were driving towards.

How many stairs are in a staircase?

So: how do you count the stairs of a staircase like this?

The basic algorithm is the same: climb until you reach the top, tracking each step you take.

The difference in implementation is in the way you track your steps: instead of waiting to evaluate the sum once you've reached the top, you evaluate as you go.

Our recursive implementation of #count-stairs from last time looked like this:

As we saw, this builds up a single computation by keeping a tally of the steps you've climbed and evaluates it once you've reached the top. For example, calling this version of #count-stairs with a staircase of four steps would build and evaluate an expression like this:

With only a slight modification, this 'non-constructive' iteration can become constructive. Let's give it a try.

If we want to keep track of how many steps we've taken at every step we take, we need a place to store that information. So the first thing to do is introduce a variable to hold the result of that computation: let's call it tally.

Since tally will hold the number of stairs we've stepped, and the number of stairs we've stepped is precisely the length of the staircase we've climbed, we now want to return our tally once we reach the top.

Finally, we add the current step to our tally and count the rest of the staircase.

This new way of 'counting the rest' is the key to an implementation that evaluates the tally as it counts, rather than deferring evaluation of the tally to when counting has stopped.

Putting it all together, our new #count-stairs function looks like this:

Quite similar to the original, no?

💡 This form of recursion is often called 'tail recursion', because the "do it again" part happens at the tail-end of the loop, and no more computation is done in the same loop.

To anyone who writes code, I highly recommend the book Structure and Interpretation of Computer Programs, by Harold Abelson and Gerald Jay Sussman. This section in particular is a great resource on the topics I'm discussing here.

What I find fascinating is how the shape of the computation changes with just these few simple modifications. Taking again a staircase of four steps to illustrate our algorithm, our computation diagram now looks something more like this:

Do you see the staircase we're climbing? 😄

Loop constructs

Iteration is useful and loops prevalent. We-Who-Code even apply the concept of iteration to applying the concept of iteration.

But depending on the tools you're using, repeatedly calling the same function isn't the only way to loop.

Many languages provide explicit "loop constructs" we can use in situations where a recursive function call may negatively impact how well our code communicates our intent to other humans and machines.

There are also times where it may not be the most mechanically efficient one, either; see SICP for a brief discussion on the performance of various compilers with respect to handling iterative code patterns.

Since these constructs are language-specific, I won't go into much more detail than to mention a few names they are commonly given, to aid you in your quest to better know your tools:

for
while
until
loop
each
do-while

Every tool has its use. Some can make your code clearer to humans or clearer to machines, and some can strike a bit of balance between both.

"Good enough to work" should not be "good enough for you." Hold yourself to a higher standard: learn a little about a lot and a lot about a little, so that when the time comes to do something, you've got a fair idea of how to do it well.

Iterating with recursion

Daniel Brady — Sat, 25 Jan 2020 20:25:37 +0000

Cover photo by Ludde Lorentz on Unsplash

How many stairs are in a staircase?

You're walking along, thinking about dinner. Should I have ramen or udon? you ask yourself.

❗ A wild staircase appears!

Thoughts of noodles recede from your mind as you consider this new development in staircase behavior, which may be worthy of a Nat Geo special.

The staircase is not too tall; you can't tell exactly how many steps there are, but you're pretty confident you can conquer them.

But...how can you be sure?

You've climbed hundreds, if not thousands, of steps in your lifetime, enough that this new set shouldn't be a problem. The secret, you know, is twofold:

Know when to stop.
Take the next step.

And you really can't climb a staircase and emerge unscathed without knowing this secret.

You recall a time when you continued to think about noodles while trying to ascend a staircase, and tripped: you'd made a mistake taking the next step.

You also recall, during this very same ascension, how it felt when you reached the top but kept on climbing because you were still weighing your noodle options: your foot came down hard and you pitched forward, nearly falling flat on your face. You'd momentarily forgotten the other secret of climbing stairs: knowing when to stop.

You eye the stairs before you, and, driven by the need to know just how many there are, you make the only reasonable decision: you clear your mind of noodles, and take the first step.

Confidence in your ability to conquer any flight of stairs, even if you've never done it, seems valuable to you in your future as a human.

But how do you incorporate counting into it? 🤔

It's simple, you realize: you merely keep a tally of the stairs you've stepped as you climb. The length of a staircase can thus be described as the value of the first stair, plus the length of the rest of the staircase.

If you've reached the top, you stop climbing, and thus stop counting. Otherwise, you can focus on stepping and summing, and wait to evaluate the tally you've kept until you've stopped.

Smiling and slightly winded, you reach the top and look at how far you've come. Full of the glow of victory, you wonder at the power of this simple algorithm. Your smile fades to a frown, however, as you remember a more pressing concern: what bowl of noodles should you have for dinner?

This approach to solving problems that can be broken down into a sequence of similar steps, where

you do something at every step
you know how to get to the next step, and
you know how to determine if you should stop stepping

is fundamental in computing science and every-day programming.

Iteration, that's what we've landed on. Doing things over and over until a particular goal is reached.

The example functions I used to illustrate the story above implemented iteration in a recursive fashion.

Recursive functions are the simplest form of programatic iteration: they are functions designed to deconstruct a problem into a linear computation of its pieces, and then evaluate it as a whole.

Thinking about it so formally can get a bit confusing. I find it easiest to grok by reading a simple recursive function aloud. Revisiting our #count-stairs example:

This can be read in plain English as:

When counting the steps of a staircase, if you're at the top, stop counting.
Otherwise, add 1 to the result of counting the rest of the staircase.

Bit of a strange loop, to be sure; but note how each time we "count the rest" of the staircase, the amount of things we are counting is getting smaller until eventually, there's nothing left to count: we've reached the top.

Let's draw out an example computation to help us see this in action:

Quite elegant, no?

Readers may notice, especially after seeing the computation diagram above, that we're arriving at the final summation without tracking the intermediate results. Good eyes, dear reader. 👏

As we've seen, this style of recursion approaches a problem wholistically and lazily: it views the solution to a problem as an aggregation of the solution to its parts, and thus doesn't compute the whole solution until it has broken it down fully.

But what happens if we fall off the stairs? What happens if we get too hungry, and need to abandon our climb in order to go eat noodles, but want to return to finish counting later? What happens if we want to text someone our current location, and tell them what step we're on?

In such situations, we care about partially computing results. At any given step, we may want to stop and reflect on how far we've come, beyond just asking "are we done yet?" And in these cases, the approach we've taken today may not be the best one for the job.

For this and other reasons, many programming languages give you access to loop constructs, and I'll talk more about those next time. 👋

Every tool has its use. Some can make your code clearer to humans or clearer to machines, and some can strike a bit of balance between both.