Forem: Nate Clark

Vanity Rust - Implementing Fn on Rust Structs

Nate Clark — Tue, 25 Feb 2020 15:38:38 +0000

The other night I was doing a coding challenge on codewars. It didn't really have anything to do with algorithms. Instead it focused on your ability to construct a specific API. On codewars, the challenge is only available in Python, JavaScript, and Ruby, but I thought it would be fun to try to solve it with Rust. It turns out that I needed to use some nightly Rust features, so I thought I'd share. The challenge can be summed up in the following line:

assert_eq!(seven(times(five())), 35);

Just from a type signature level, we can determine a few things:

We need some "number" functions (seven and five) and some "operator" functions (times).
The "number" functions need to be able to be called with 0 arguments or 1 argument.
The "operator" functions will always be called with 1 argument

In this post, I want to focus on how you would go about implementing the number functions.

Depending on your language of choice you probably end up googling one of two topics: function overloading, or default arguments. If you were to do so, you would quickly find several kind posts about how Rust doesn't support either of these. People will suggest accepting an Option argument or simply creating multiple functions. These are both great suggestions for real software, but this is about code golfing. So instead... we're going nightly.

Nightly Rust

$ rustup override set nightly

But we're not just doing it for fun, we're doing it because we need two specific nightly features: unboxed_closures (docs), and fn_traits (docs).

Unboxed Closures

unboxed_closures gives us the ability to reach into a lower level of function calling logic. This will feel somewhat similar to defining a variadic function (e.g. using arguments or rest parameters in JavaScript). Let's write a simple add function in Rust normally, and then using unboxed_closures.

// normal
fn add(x: isize, y: isize) -> isize {
    x + y
}

#![feature(unboxed_closures)]
// with unboxed_closures
extern "rust-call" fn add(args: (isize, isize)) -> isize {
    args.0 + args.1
}

However, there is a catch. The unboxed_closures version of add technically has a single argument that is a tuple. This is important to us, because we have to use unboxed_closures to implement a Fn trait for our struct.

Implementing Fn

Let's go ahead and dump out a struct to box up our number. In just a minute we'll write the required code to treat our structs like functions.

struct Number {
    number: isize
}

const five: Number = Number { number: 5 };
const seven: Number = Number { number: 7 };

Okay, so we have these structs that box up an integer. Now we need to let Numbers act like functions. Additionally, we need it to act like a different function depending on the arguments (function overloading). We'll do this by implementing the FnOnce trait for Number. FnOnce requires that we provide an Output type and a call_once method that is an unboxed_closure. Considering we need it to be overloaded, we will provide an implementation for each argument configuration. Note: In my second implementation, I use an argument type of
Operation. I'm not defining Operation here, but it is part of how solved the original coding challenge.

#![feature(fn_traits, unboxed_closures)]

// When called with 0 arguments
impl FnOnce<()> for Number {
    type Output = isize;
    extern "rust-call" fn call_once(self, args: ()) -> Self::Output {
        self.number
    }
}

// When called with one argument of type `Operator`.
impl FnOnce<(Operation,)> for Number {
    type Output = isize;
    extern "rust-call" fn call_once(self, args: (Operation,)) -> Self::Output {
        // Do something with self and the Operation
    }
}

Recap

assert_eq!(seven(times(five())), 35);
                 ^^^^^ Not found in this scope

Congrats! We can now create a struct and treat it like a function with overloading. Way simpler than doing it in JavaScript/Python/Ruby! Of course I'm joking, but I still think Rust is super interesting. More dynamic languages will always win at code golf, but Rust is crazy fast, crazy safe, and is a very positive movement in programming languages.

Inspecting function annotations in Python

Nate Clark — Mon, 03 Feb 2020 16:40:10 +0000

For this article, I'm just going to spend some time in the python shell. Please let me know if this is helpful or terrible 😄.

>>> import inspect
>>> # Let's learn a little about the inspect module
>>> inspect.__doc__.splitlines()[0]
'Get useful information from live Python objects.'

>>> inspect.signature.__doc__
'Get a signature object for the passed callable.'

>>> inspect.Signature.__doc__.splitlines()[0]
'A Signature object represents the overall signature of a function.'

>>> # Let's inspect a simple function
>>> def add(x, y):
...     return x + y

>>> add_sig = inspect.signature(add)
>>> add_sig
<Signature (x, y)>

>>> add_sig.parameters['x']
<Parameter "x">

>>> add_sig.parameters['x'].annotation
<class inspect._empty>

>>> add_sig.return_annotation
<class inspect._empty>

>>> # If the function has more information
>>> # we can inspect more information
>>> def typed_add(x: int, y: float) -> float:
...     return x + y

>>> typed_add_sig = inspect.signature(typed_add)
>>> typed_add_sig.parameters['x'].annotation
<class int>

>>> typed_add_sig.return_annotation
<class float>

>>> # Let's pretend that we're pytest
>>> # and implement a baby version of fixtures
>>> def test_42_is_the_answer(the_answer):
...     assert the_answer == 42

>>> fixtures = {"the answer": 42}

>>> def fake_test_run(test_function):
...     test_function_parameters = inspect.signature(test_function).parameters.values()
...
...     if any(parameter.name not in fixtures for parameter in test_function_parameters):
...         raise Exception(f"Test function expected expected fixture named {name}")
...
...     arguments_to_pass = {
...         parameter.name: fixtures[parameter.name]
...         for parameter in test_function_parameters
...     }
...
...     try:
...         test_function(**arguments_to_pass)
...     except AssertionError as ex:
...         print("Fail: Test Failed")
...         print(ex)
...     except Exception as ex:
...         print("Error: Something bad happened")
...         print(ex)
...     else:
...         print("Pass!")

>>> fake_test_run(test_42_is_the_answer)
Pass!

>>> fixtures["the_answer"] = 2345

>>> fake_test_run(test_42_is_the_answer)
Fail: Test Failed

>>> # Goal!
>>> # Our test function is *magically* getting passed an argument
>>> # based on our function signature.

ADTs in OOP - FTW or WTF?

Nate Clark — Sat, 18 Jan 2020 14:16:09 +0000

A lot of times it might feel like some development communities are far apart. This is totally understandable. We are all doing our best, and we don't have the time or energy to stay up to date with every corner of the development world. In this post I want to bring the typed functional world and the object oriented world a little closer together. To do so, let's take a small snippet adapted from the ReasonML documentation and dissect it through porting to TypeScript. You can play with this snippet on the try reason website.

type account =
  | None
  | Instagram(string)
  | Facebook(string, int);

let greeting = (account) =>
  switch (account) {
  | None =>
        "Hi!"
  | Facebook(name, age) =>
        "Hi " ++ name ++ ", you're " ++ string_of_int(age) ++ " years old."
  | Instagram(name) =>
        "Hello " ++ name ++ "!"
  };

let myAccount = Instagram("nate")
let myGreeting = greeting(myAccount)
Js.log(myGreeting)

What is Reason Doing Here

I think there are 2 big ideas required to understand this code: variants, and pattern matching.

It is worth mentioning that some communities call variants by another name. You might hear terms like "Algebraic Data Type", "Sum Type", or "Union Type". These are all roughly the same thing as Reason's variant.

The `account` Variant

type account =
  | None
  | Instagram(string)
  | Facebook(string, int);

This code introduces one new type, account, and three ways to construct a value of that type, None, Instagram, and Facebook. The None constructor doesn't require an data. The Instagram constructor requires a string. The Facebook constructor requires a string and an int. Hmmm... new type... constructors... Let's eagerly jump to an object oriented version of this code.

class Account {
  name?: string;
  age?: number;
  type: string;

  private constructor(name?: string, age?: number, type: string) {
    this.name = name;
    this.age = age;
    this.type = type;
  }

  static None() {
    return new this(undefined, undefined, "None");
  }

  static Instagram(name: string) {
    return new this(name, undefined, "Instagram");
  }

  static Facebook(name: string, age: number) {
    return new this(name, age, "Facebook");
  }
}

Account type introduced: ✔
Can construct a None account with no data: ✔
Can construct an Instagram account with a name: ✔
Can construct a Facebook account with a name and age: ✔

This is looking fair enough so far. As far as representing the Reason code goes, we're well on our way. Now let's take a look at that greeting function.

Pattern Matching in `greeting`

let greeting = (account) =>
  switch (account) {
  | None =>
        "Hi!"
  | Facebook(name, age) =>
        "Hi " ++ name ++ ", you're " ++ string_of_int(age) ++ " years old."
  | Instagram(name) =>
        "Hello " ++ name ++ "!"
  };

The heart of this code is branching based on the type of the account.

If it is a None account, we simply greet with "Hi!".
If it is a Facebook account, we greet based on the name and age.
If it is an Instagram account, we greet based on the name.

This sounds like a pretty mechanical translation as well. And since Reason co-opted switch for pattern matching, we can even just use a regular JavaScript switch. However, instead adding a greeting function, let's make it a method on Account.

class Account {
  // ... our Account definition from before ...

  greeting(): string {
    switch (this.type) {
      case "None":
        return "Hi!";
      case "Facebook":
        return `Hi ${this.name}, you're ${this.age} years old.`;
      case "Instagram":
        return `Hello ${this.name}!`;
    }
  }
}

So, at this point we are almost finished. Let's wrap this up with console.log.

// original reason code in comment above each line

// let myAccount = Instagram("nate");
let myAccount = Account.Instagram("Nate");

// let myGreeting = greeting(myAccount);
let myGreeting = myAccount.greeting();

// Js.log(myGreeting);
console.log(myGreeting);

Not Quite

You might be screaming at me right now. I know that typescript is screaming at me right now:

Function lacks ending return statement and return type does not include 'undefined'.
- The TypeScript Compiler

Our problem is familiar to anyone who has done typed programming. Our method promises to return a string, but we did not make good on that promise. If this.type is something other than "None", "Instagram", or "Facebook", our code will happily return undefined. We could return some dummy or throw an exception, but I think we are missing out on some connection between algebraic data types and object oriented programming. Let's take a step back.

Another Way

Our first red flag should have been that we were branching based on some sort of type information. Object oriented programming languages have a very special tool for branching based off type information. It is called method dispatch. We should not write the branch, it is already built into the language. We just have to model our problem appropriately. If we are going to do method dispatch, we want greeting to go to the right place. Instead of writing the branch ourselves, we need three different greeting methods for the language to choose from. If we want our Account type to look the same from the outside, we will have to create three subclasses. A couple nice things will fall out of this. Let's look.

interface Account {
  greeting(): string;
}

class None implements Account {
  greeting(): string {
    return "Hi!";
  }
}

class Instagram implements Account {
  name: string;

  constructor(name: string) {
    this.name = name;
  }

  greeting(): string {
    return `Hello ${this.name}!`;
  }
}

class Facebook implements Account {
  name: string;
  age: number;

  constructor(name: string, age: number) {
    this.name = name;
    this.age = age;
  }

  greeting(): string {
    return `Hi ${this.name}, you're ${this.age} years old.`;
  }
}

Let's break down what happened.

We no longer have an Account class, instead we have an Account interface. This makes a lot of sense. The reason account type was really just a label for either the None, Instagram, and Facebook account data structures. If we never pattern matched on the account to pull out the underlying data, it was just a black box. Once we pattern matched, we knew that we were dealing with one of those structures, and we could actually do work.

We also got rid of those ?s! This is a great sign because name and age are not truly optional. If the account is a Facebook account, they are always required. By making each variant of account into its own class, we get to model the data required by each variant on its own. Then we use implements Account to tie them all back together.

ADT -> OOP

So we have a bit of a formula here. It is a simple one, and it will need some more work once you use more interesting forms of pattern matching. However, I think it provides a lot of intuition.

You can think about each variant/algebraic data type/union as an interface.
Each constructor is a class that implements that interface.
If a function simply pattern matches on the constructor type, it is a method on the interface.

In this example:

We turned type account into interface Account
We turned None, Instagram, and Facebook into classes that implement Account
We turned greeting into a method on the Account interface.

Learning More

These two courses really cemented a lot of this comparative stuff for me. I think they are pretty great, and I highly recommend them.

Thanks

Hopefully this article has helped you cross that boundary between statically typed, functional languages and class based, object oriented languages a little smoother.

Thanks for reading!

Dependency Injection From First Principles

Nate Clark — Tue, 14 Jan 2020 03:02:40 +0000

If you are starting to learn web development, you have probably run into dependency injection.

If you are working with Spring it might look like this:

class MyClass {
    @Autowired
    public MyClass(MyDependency myDependency) { ... }
}

Angular looks almost identical:

@Component(...)
class MyClass {
  constructor(myDependency: MyDependency) {
    ...
  }
}

Or maybe you are testing in Python with pytest and it looks like this:

def test_user_can_purchase_a_product(client):
    ...

Dependency injection comes in a ton of contexts and varieties. In this post I want to explore "What Is Dependency Injection?", "Why Do We Do It?", and "How Do We Do It?".

What Is Dependency Injection?

At its heart, dependency injection is about moving code out of a particular function/class and into an argument to that function/class.

def login(username, password):
    if check_credentials(username, password):
        ...
    else:
        ...

It is pretty plain to see that check_credentials is used by login. In other words, login depends on check_credentials. login has other dependencies though. It also depends on username and password. However, there is an important distinction between check_credentials and username/password.

The username and password dependencies are provided as an argument. If I want to change username or password, I call it with different arguments.

The check_credentials dependency is hard-coded inside login. If I want to change check_credentials, I rewrite login. This might be totally fine. You might never want to change it. But if you do, the most natural place would be to accept it as an argument.

def login(username, password, check_credentials):
    if check_credentials(username, password):
        ...
    else:
        ...

Now, whenever login gets called, you just provide the appropriate function for check_credentials. login still depends on check_credentials, but it is no longer a hard-coded dependency. Unfortunately, we've now made every call site a little more annoying. Now, every time we call login we have to provide an appropriate check_credentials. We'll talk about making it less annoying in a second, but first let's talk about why we do this at all.

Why Do We Do It?

From my exposure, we do it for three reasons: testing, decoupling, and we have to. I'm certain my perspective is limited, but I'll limit myself to discussing these three reasons.

Testing

If you are using some mocking tool to replace a class or function while testing, you are changing that dependency. If that dependency were provided as an argument, you would not need the mocking tool.

from unittest.mock import patch

def test_login_success():
    # changing the dependency through a mocking tool
    with patch('check_credentials', return_value=True):
        login('admin', 'password123')
        ...

def test_login_success():
    def always_pass(username, password):
        return True

    # changing the dependency by providing a different argument
    login('admin', 'password123', always_pass)
    ...

I'm not here to argue for or against mocking, but it is valuable to realize that both examples are mocking. Even though the second example doesn't use any fancy tools to mess with login before it runs, it is still providing a mock implementation. The example where we provide always_pass as an argument just requires less heavy lifting.

Decoupling

I don't want to spend a ton of time on this topic, but I think it is important to acknowledge. The style of programming allowed by dependency injection supports a more loosely coupled architecture. As a proof of concept we can look at our login function. If login has a hard coded dependency on check_credentials, it is coupled to the code inside it. That means login is coupled check_credentials (and its dependencies). So by moving check_credentials into an argument, we move ourselves towards the dependency inversion principle. We are now depending on an interface instead of an implementation. When it is hard coded, we are depending on the specific implementation. When it is an argument we are depending on however we use it. In the case of check_credentials, we are now depending on it being a function with the signature (str, str) -> bool.

We Have To

I think the other most common reason is that your tool of choice makes you. It is hard for me to think of a modern web development framework that doesn't contain some sort of dependency injection solution. Instead of trying to highlight a bunch of examples, I want to give you an idea of what to look for in your framework of choice.

How Do We Do It?

Let's go back to the code that we don't want to write.

def some_function(...):
    login(username, password, real_check_credentials)

We don't want to write real_check_credentials all over the place in our application. The good news is that you usually don't have to. Most contexts where you hear people use the term "dependency injection" are working inside a dependency injection container. A dependency injection container is the central piece of machinery that allows for a more pleasant experience with this style of programming. If you see some of the following things, you might be inside a dependency injection container:

A way of registering dependencies:
- @Injectable (Angular)
- @pytest.fixture (pytest)
- services.AddScoped (ASP.NET)
A way of hooking into a dependency context or requesting dependencies:
- @Autowired (Spring)
- @Component (Angular)
- arguments you don't provide (pretty much all of them)
The framework calls the function or constructs the object for you

Looking at those three clues, we can get a good idea of what must be going on behind the scenes. There must be some registry of dependencies being maintained and provided on demand. The third bullet outlines the trick to making it all work. If you were to construct the object or call the function yourself, you would have to provide the argument. However, if the framework is doing the work it can provide the required dependencies at the appropriate time. This is the dependency injection container at work. You register dependencies, request dependencies, and provide some sort of class or function to be called where that registry is available.

Thanks

Hopefully this provides a helpful look at the basics of dependency injection. Dependency injection is all over the place, and I remember being disoriented when I first worked with it.

Thanks for reading!

Getting My First PR Merged

Nate Clark — Fri, 08 Mar 2019 20:21:06 +0000

I recently had my first non-trivial PR merged into an open source project, and I was hoping that my story could help you feel empowered to contribute as well.

Open Source will find you

First, I want you to feel empowered, not indebted, to get involved. Despite what we may feel (or be told), you can be a perfectly good developer without ever making an open source contribution.

I was helping my students debug some DB code, and I noticed a specific error message that I thought could be improved.

(sqlalchemy.exc.InvalidRequestError) A value is required for bind parameter 'id' [SQL: 'select\\n    *\\nfrom\\n     reviews\\nwhere\\n    id = %(id)s'] (Background on this error at: http://sqlalche.me/e/cd3x)

To be clear, I think this is a good error message. It even includes a shortened link for finding more help online. I just thought it could be improved.

Scratch an itch if you have one. If you feel like you don't have anything to contribute to a project. That. Is. OK. You are already doing good work.

Baby steps

I had two primary pain points:

Terminal wrapping is no fun. Maybe a little work could be done to split the error message up more gracefully over multiple lines.
The relevant SQL statement looks a little different than it was written. I thought some work could be done to make the SQL more easily recognizable.

So the above example would have turned into something like the following:

(sqlalchemy.exc.InvalidRequestError) A value is required for bind parameter 'id'
[SQL: (select
    *
from
    reviews
where
    id = %(id)s)]
(Background on this error at: http://sqlalche.me/e/cd3x)

Awesome, I have a clearly-scoped itch to scratch. Finding the code responsible didn't take too much time, and it was so nice and small. I had to change two tiny lines.

        details = [self._message(as_unicode=as_unicode)]
        if self.statement:
-           details.append("[SQL: %r]" % self.statement)
+           details.append("[SQL: %s]" % self.statement)
            if self.params:
                params_repr = util._repr_params(self.params, 10)
                details.append("[parameters: %r]" % params_repr)
        code_str = self._code_str()
        if code_str:
            details.append(code_str)
-       return " ".join(["(%s)" % det for det in self.detail] + details)
+       return "\n".join(["(%s)" % det for det in self.detail] + details)

I had to change a %r to a %s and a " " to a "\n".

It might take a little work to find the relevant code, but a novice programmer can make this change.

Nothing is that simple

The project in question, SqlAlchemy (https://github.com/sqlalchemy/sqlalchemy/), is a popular Python project that supports Python 2 and 3. So I hit a snag related to cross-version support that I wasn't really prepared to solve on my own. Also, a good handful of tests broke for such a small change. Take a peek at the final PR:

Issue #4500 : Change StatementError formatting (newlines and %s) #4501

natec425 posted on Feb 17, 2019

This changes the error formatting for StatementError in two ways:

Break each error detail up over multiple newlines instead of spaced out on a single line. Hopefully, this helps readers scan the error message more easily.
Change the SQL representation in the message to use str (%s) instead of the current behavior that uses repr (%r). This should help readers recognize the structure of their SQL, particularly if it is a multiline SQL statement. In the multiline case, the SQL would be printed over multiple lines instead of printing an escaped "\n".

Fixes: #4500

This pull request is:

[X ] A short code fix
- please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted.
- Please include: Fixes: #<issue number> in the commit message
- please include tests. one line code fixes without tests will not be accepted.

View on GitHub

That small diff ballooned a bit.

You will fail a CI build. It is OK.

But there is help

Thankfully the project's maintainer was super helpful and friendly every step of the way. I didn't feel comfortable digging into the python2-3 issues, so he stepped in and sorted that piece out. If it was up to me, I would be left with "So when is SqlAlchemy dropping support for python2 😄?". So that brings me to my last takeaway.

Solving the problem is not the goal.
Being a part of the team is the goal.

Repeating myself

You can contribute. Don't feel like you need to. Your own work is important.
If you notice an opportunity, remember that you can do it.
It will probably be difficult in ways that you didn't expect.
You are not alone. Hitting a road bump is part of being on the team.

Forem: Nate Clark

Vanity Rust - Implementing Fn on Rust Structs

Nightly Rust

Unboxed Closures

Implementing Fn

Recap

Inspecting function annotations in Python

ADTs in OOP - FTW or WTF?

What is Reason Doing Here

The account Variant

Pattern Matching in greeting

Not Quite

Another Way

ADT -> OOP

Learning More

Thanks

Dependency Injection From First Principles

What Is Dependency Injection?

Why Do We Do It?

Testing

Decoupling

We Have To

How Do We Do It?

Thanks

Getting My First PR Merged

Open Source will find you

Baby steps

Nothing is that simple

Issue #4500 : Change StatementError formatting (newlines and %s) #4501

But there is help

Repeating myself

The `account` Variant

Pattern Matching in `greeting`