Forem: Krzysztof

The Business Value of Refactoring and Testing

Krzysztof — Tue, 06 Aug 2024 08:34:13 +0000

Over the years I had to deal with applications and system that have a long history of already being "legacy".
On top of that I met with clients/product owners that never want you to spend time refactoring. Some of them will even be aware that things like automated tests exist and will specifically tell you to not write them.

On one hand I can understand them. They do not understand code (even when they think so), they don't understand how vulnerable it is, but they understand how expensive it can be.

Doing an MVP, greenfield project often comes with restrictions on time and budget, so those kind of projects are the most vulnerable, and people who want them done, are usually in need of convincing that tests are important, and we should not write code, that requires refactoring immediately after it is done. That means that we need more time, and if we can't have that, we need to rethink our partnership.

Same goes for legacy projects, with terrible or bad codebases "that just work". They gather more technical debt than money, and there is never time for refactoring, specially since none (or not enough) of the code is covered with tests. Refactoring will be too time consuming when they are no tests to make sure that you are not just breaking stuff, instead of making it better, right?

Who is this for

This short post is aimed at developers mostly, and particularly at those, who put up with the behaviour of clients described above. I've historically had some successes and failures when it comes to convincing people that refactoring is needed, I had to persuade clients that had some technical knowledge, that tests are actually beneficial and time spent refactoring is time spend on making more money.

The point of this

Based on a Guilded Rose Kata I want to present a situation, that you can either use as an example, or learn from or at least take some motivation from.

The Kata

Guilded Rose Kata is a rather popular programming exercise, where you are given a bad piece of code "that just works", with no tests. You also get the general description of how everything works and what are you supposed to do (and what you cannot do). Here is the whole description:

Hi and welcome to team Gilded Rose. As you know, we are a small inn with a prime location in a prominent city ran by a friendly innkeeper named Allison. We also buy and sell only the finest goods. Unfortunately, our goods are constantly degrading in quality as they approach their sell by date. We have a system in place that updates our inventory for us. It was developed by a no-nonsense type named Leeroy, who has moved on to new adventures. Your task is to add the new feature to our system so that we can begin selling a new category of items. First an introduction to our system:

All items have a SellIn value which denotes the number of days we have to sell the item
All items have a Quality value which denotes how valuable the item is
At the end of each day our system lowers both values for every item
Pretty simple, right? Well this is where it gets interesting:

Once the sell by date has passed, Quality degrades twice as fast
The Quality of an item is never negative
“Aged Brie” actually increases in Quality the older it gets
The Quality of an item is never more than 50
“Sulfuras”, being a legendary item, never has to be sold or decreases in Quality
“Backstage passes”, like aged brie, increases in Quality as it’s SellIn value approaches; Quality increases by 2 when there are 10 days or less and by 3 when there are 5 days or less but Quality drops to 0 after the concert

We have recently signed a supplier of conjured items. This requires an update to our system:

“Conjured” items degrade in Quality twice as fast as normal items

Feel free to make any changes to the UpdateQuality method and add any new code as long as everything still works correctly. However, do not alter the Item class or Items property as those belong to the goblin in the corner who will insta-rage and one-shot you as he doesn’t believe in shared code ownership (you can make the UpdateQuality method and Items property static if you like, we’ll cover for you).

The process

If you are dropped in this kind of situation in real life, and see this code (using ruby here):

class GildedRose

  def initialize(items)
    @items = items
  end

  def update_quality()
    @items.each do |item|
      if item.name != "Aged Brie" and item.name != "Backstage passes to a TAFKAL80ETC concert"
        if item.quality > 0
          if item.name != "Sulfuras, Hand of Ragnaros"
            item.quality = item.quality - 1
          end
        end
      else
        if item.quality < 50
          item.quality = item.quality + 1
          if item.name == "Backstage passes to a TAFKAL80ETC concert"
            if item.sell_in < 11
              if item.quality < 50
                item.quality = item.quality + 1
              end
            end
            if item.sell_in < 6
              if item.quality < 50
                item.quality = item.quality + 1
              end
            end
          end
        end
      end
      if item.name != "Sulfuras, Hand of Ragnaros"
        item.sell_in = item.sell_in - 1
      end
      if item.sell_in < 0
        if item.name != "Aged Brie"
          if item.name != "Backstage passes to a TAFKAL80ETC concert"
            if item.quality > 0
              if item.name != "Sulfuras, Hand of Ragnaros"
                item.quality = item.quality - 1
              end
            end
          else
            item.quality = item.quality - item.quality
          end
        else
          if item.quality < 50
            item.quality = item.quality + 1
          end
        end
      end
    end
  end
end

class Item
  attr_accessor :name, :sell_in, :quality

  def initialize(name, sell_in, quality)
    @name = name
    @sell_in = sell_in
    @quality = quality
  end

  def to_s()
    "#{@name}, #{@sell_in}, #{@quality}"
  end
end

This is obviously rather hard to follow. Mapping this to real world description of the system is not easy to follow.

Ruby is often praised (or hated) for its "english-like" syntax, so mapping almost a bullet point description to if code shouldn't be too hard, but the amount of nesting and conditionals do make me lose track of things. Most lines are not "stand-alone" they don't necessarily make sense on their own, and I have to constantly start reading the whole method again, or go back to the description.

We want something that "flows", code that makes it easy to add more features or change existing ones. Client wants that too, cause that means in the future, he has to spend less money on you.

It's a simple rule in IT, that money people will always understand:

The better the code, the less time it takes to add to it or to change it

This is where we have to start the discussion. The money side of things. The time it takes.

When discussing refactoring, never, ever approach it from the angle of perfectionist or even the POV of a engineer. Try to take a far more pragmatic, business and materialistic approach.

Don't say:

This code is so bad it makes me feel bad about being a programmer

Say:

This code is so bad, that it will cost you more money if you ever want to change anything.

There is no:

This does not follow any of the best practices

There is just:

This is so insufficient that it will scare away customers cause it will work worse and worse. All decent programmers will demand more money to work on it as time goes on

Always focus on money. Clients and product owners don't care if you think about them, but they do care if you think about their money, and how to make them have more of it.

This is mine main advice if you have to talk to clients that have limited budget. Don't focus on technology, don't try to show them how great of a engineer you are, don't explain how good code should be written, they don't care. Tell them why stuff is so expensive, why small changes cost them a lot of money, and what can you do to help them save it now and in the future.

Remember that coding is often mostly reading, rather than writing. Clients and non-tech people will often not be aware of it. If they are made conscious that first you have to understand the code you need to change, they might get a bit different perspective, and understand that if code is bad, it means it is hard to read, understand and change. Harder means more time consuming. More time == more money

This is how you can convince them to spend a few hours on refactoring that in the future will save them days. Time is money, so focus on those two things. How long will a feature take if code looks like that, and how long will it take, if you change it to look the other way.

Only once you get the green light on that, you can start talking about tests.

People that pay you will mostly care about time. How long will a task take? Why so long, and just make it faster, cut some corners!

They often think (sometimes subconsciously), that programmers only care about writing some awesome technological code. They might consider developers as people who don't think about money. You need to make sure they know, that you care about how much they spend on you. Once they know that you are aware of the financial side of things, they might trust your judgement more.

How to talk about tests

This is the most difficult thing, cause if you are in a position where you have to talk about tests with the client, you are already in a bad spot.

It is a technical detail, something clients, and business people in general should not worry about. It should be implied, that we only deliver code that we think is good, and good code is tested.

However, we have seen some other situations. Mostly in cases, where the client is a former technical person (or still slightly technical) and they know what tests are, but think they just come with more time, and provide little value.

How do you argue with that, when there is still a lot of programmers that think the same?

There are 4 main things to try, and if those fail, you can either accept your damned faith, or look for a different project/job/client.

Tests are a mark of something being actually done. If you write tests for a feature (which is easier than writing the whole feature), you get a unit of measurement of that something actually being done. Also the more of those you write, the better you get with estimations (mapping out how feature relates to tests and that to implementation estimation is a topic for a different time, but in general, it comes with practice).
When refactoring, it not only makes you more confident in making changes, it actually secures the safety of those changes. If you convinced them to refactoring, it is crucial to make sure, they understand, that refactor aims to create a good code out of a bad one, and good code is tested. When you are confident in changes and safe in making them, they take less time.
Regression is something that can cost a lot of money. Seemingly unrelated change, breaks something that seems completely disjointed from it. Well known tale in IT, one that we can avoid to spend money on, if we have good tests.
If they ever change programmer, they will have easier, quicker time onboarding them into the code that is well tested. That will mean less money spend on time needed for them to understand the codebase and make the same mistakes someone with no tests would do.

In general all the above aim to drive home one main point:

Tests save time, not add more of it

The better the testing suite, the easier and safer the changes to the actual code are. This means that code will not break as often and will be easier to expand on. Easier is faster. Faster is cheaper.

MONEY

Look like this on your webacm calls with the client

Make sure you mention it a lot.

The code

Lets actually solve the kata, ok? I will show you my process of testing and refactoring those kind of "features", "problems", or whatever we want to call it.

First of I map the requirements to a spec, in this case, RSpec and Ruby make it super easy and clean, since it is like reading english (almost).

Here is how I mapped the requirements to the spec (I start with just the descriptions, without the code, I only did a simplistic test for the Item class since I can't change it anyway due to requirements):

Initial spec

describe GildedRose do

  it 'does not change the name' do
    items = [Item.new('foo', 0, 0)]
    GildedRose.new(items).update_quality
    expect(items[0].name).to eq 'foo'
  end

  describe 'Item' do
    it 'should have a name, sell_in, and quality' do
      item = Item.new('foo', 0, 0)
      expect(item.name).to eq 'foo'
      expect(item.sell_in).to eq 0
      expect(item.quality).to eq 0
    end

    it '#to_s' do
      item = Item.new('foo', 0, 0)
      expect(item.to_s).to eq 'foo, 0, 0'
    end
  end

  context 'with quality' do
    it 'degrades twice as fast if sell_in has passed (negative value)' do

    end

    it 'cannot be negative' do

    end

    it 'gets higher with passage of time for Aged Brie' do

    end

    it 'cannot be higher than 50' do

    end

    it 'is always 80 for Sulfuras' do

    end

    it 'decreases by 1 for all other items' do

    end

    context 'with Backstage passes' do
      it 'increases by 2 when sell_in is 10 or less' do

      end

      it 'increases by 3 when sell_in is 5 or less' do

      end

      it 'drops to 0 after the concert' do

      end

    end

    context 'with conjured items' do
      it 'degrades twice as fast as normal items' do

      end
    end
  end

  context 'with sell_in' do
    it 'never decreases for Sulfuras' do

    end

    it 'decreases by 1 for all other items' do

    end
  end
end

That makes it easy to track all the requirements. When I see that I haven't missed anything I start doing the actual test, no changes to the code at this point.

First full spec

Here I have the contents of the test, it has 16 examples, 14 pass, 2 fail, since they are specs for the new feature not yet implemented.

With that in hand I can see that this messy code does indeed work. I still have no clear idea how to change it to add the new stuff and not blow things up, but at least I can easily experiment.

describe GildedRose do

  it 'does not change the name' do
    items = [Item.new('foo', 0, 0)]
    GildedRose.new(items).update_quality
    expect(items[0].name).to eq 'foo'
  end

  describe 'Item' do
    it 'should have a name, sell_in, and quality' do
      item = Item.new('foo', 0, 0)
      expect(item.name).to eq 'foo'
      expect(item.sell_in).to eq 0
      expect(item.quality).to eq 0
    end

    it '#to_s' do
      item = Item.new('foo', 0, 0)
      expect(item.to_s).to eq 'foo, 0, 0'
    end

  end

  context 'with quality' do
    it 'degrades twice as fast if sell_in has passed (negative value)' do
      items = [Item.new('foo', -1, 10)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 8
    end

    it 'cannot be negative' do
      items = [Item.new('foo', 0, 0)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 0
    end

    it 'gets higher with passage of time for Aged Brie' do
      items = [Item.new('Aged Brie', 0, 0)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 2
    end

    it 'cannot be higher than 50' do
      items = [Item.new('Aged Brie', 0, 50)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 50
    end

    it 'is always 80 for Sulfuras' do
      items = [Item.new('Sulfuras, Hand of Ragnaros', 0, 80)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 80
    end

    it 'decreases by 1 for all other items' do
      items = [Item.new('foo', 0, 1)]

      GildedRose.new(items).update_quality

      expect(items[0].quality).to eq 0
    end

    context 'with Backstage passes' do
      it 'increases by 2 when sell_in is 10 or less' do
        items = [Item.new('Backstage passes to a TAFKAL80ETC concert', 10, 10)]

        GildedRose.new(items).update_quality

        expect(items[0].quality).to eq 12
      end

      it 'increases by 3 when sell_in is 5 or less' do
        items = [Item.new('Backstage passes to a TAFKAL80ETC concert', 5, 10)]

        GildedRose.new(items).update_quality

        expect(items[0].quality).to eq 13
      end

      it 'drops to 0 after the concert' do
        items = [Item.new('Backstage passes to a TAFKAL80ETC concert', 0, 10)]

        GildedRose.new(items).update_quality

        expect(items[0].quality).to eq 0
      end
    end

    context 'with conjured items' do
      it 'degrades twice as fast as normal items' do
        items = [Item.new('Conjured Mana Cake', 1, 10)]

        GildedRose.new(items).update_quality

        expect(items[0].quality).to eq 8
      end

      it 'degrades twice as fast also when sell_in has passed' do
        items = [Item.new('Conjured Mana Cake', 0, 10)]

        GildedRose.new(items).update_quality

        expect(items[0].quality).to eq 6
      end
    end
  end

  context 'with sell_in' do
    it 'never decreases for Sulfuras' do
      items = [Item.new('Sulfuras, Hand of Ragnaros', 0, 80)]

      GildedRose.new(items).update_quality

      expect(items[0].sell_in).to eq 0
    end

    it 'decreases by 1 for all other items' do
      items = [Item.new('foo', 1, 1)]

      GildedRose.new(items).update_quality

      expect(items[0].sell_in).to eq 0
    end
  end
end

So with that, the refactor begins (only showing the part of the code that matter for brevity)

First thing is running rubocop to make it a bit cleaner, but it still leaves me with a mess. My first attempt is anything to make it more readable. Since most of the business logic is based on the name quality of the Item I see an opportunity to redo the if statements to case.

At least for the most part. Anyway, this is what the first version looks like:

  def update_quality
    @items.each do |item|
      item.sell_in -= 1 unless item.name == 'Sulfuras, Hand of Ragnaros'
      change_quality(item)
    end
  end

  private

  def change_quality(item)
    case item.name
    when 'Aged Brie'
      item.quality += 1
    when 'Backstage passes to a TAFKAL80ETC concert'
      if item.sell_in.negative? || item.sell_in.zero?
        item.quality = 0
        return
      end
      item.quality += case item.sell_in
                      when 1..5
                        3
                      when 6..10
                        2
                      else
                        1
                      end
    when 'Sulfuras, Hand of Ragnaros'
      item.quality = 80
      return
    when 'Conjured Mana Cake'
      item.quality -= item.sell_in.negative? ? 4 : 2
    else
      item.quality -= item.sell_in.negative? ? 2 : 1
    end
    item.quality = 50 if item.quality > 50
    item.quality = 0 if item.quality.negative?
  end

It is still messy, it still has a lot of if statements, but at least at first glance you can find places that hold the whole logic for a specific item. With nested ifs, that was harder to follow.

This first version of refactoring takes around 10-15 minutes. I did it by simply deleting all the ifs, and following the spec step by step, handling each case. That made it super easy to map the spec requirements to the case code and make the code pass.

However at the end of the day, it is barely better than the original. Adding new features seems like adding just more when statements, but there is still some ending logic that complicates stuff, and we end up nesting cases inside and there is still some leftover conditionals.

I myself am not a big opponent of conditional statements, there is a lot of programmers in Ruby that just grin and twist when they see an if. They get angry when they remember 102432 design patterns that could be used instead of an if, but I am too stupid for that, so I don't mind them from time to time, especially when using ternary operators, that just ensure simple 0/1 kind of result.

To adhere to this obsession though, and due to my own dislike of the case implementation, I will try something else.

I have been experimenting with Elixir programming language and it has a feature called pattern matching. Not just some feature by the way, it plays a major role in that language. Ruby has had it too for some time, although it is not as advanced and mature as in Elixir (and will never be due to the nature of both languages), but hey, it works, and seems to be a match made in heaven for cases like this.

Final refactor

This refactor took another 15 minutes, I also added one more spec, cause refactoring with the specs made me realise I need to handle one more case (see? more value of refactoring and testing).

This shows the whole code this time.

class GildedRose
  def initialize(items)
    @items = items
  end

  def update_quality
    @items.each do |item|
      update_item(item, item.sell_in, item.name)
    end
  end

  private

  def update_item(item, *pattern)
    case pattern
    in [_, 'Sulfuras, Hand of Ragnaros']
      set_quality(item, 80)
    in [_, 'Aged Brie']
      age_item(item)
      increase_quality(item, 1)
    in [6..10, 'Backstage passes to a TAFKAL80ETC concert']
      age_item(item)
      increase_quality(item, 2)
    in [1..5, 'Backstage passes to a TAFKAL80ETC concert']
      age_item(item)
      increase_quality(item, 3)
    in [Integer => sell_in, 'Backstage passes to a TAFKAL80ETC concert']
      age_item(item)
      sell_in.negative? || sell_in.zero? ? set_quality(item, 0) : increase_quality(item, 1)
    in [_, 'Conjured Mana Cake']
      age_item(item)
      item.sell_in.negative? ? degrade_quality(item, 4) : degrade_quality(item, 2)
    in [Integer, String]
      age_item(item)
      item.sell_in.negative? ? degrade_quality(item, 2) : degrade_quality(item, 1)
    end
  end

  def degrade_quality(item, level)
    item.quality = [item.quality - level, 0].max
  end

  def increase_quality(item, level)
    item.quality = [item.quality + level, 50].min
  end

  def set_quality(item, quality)
    item.quality = quality
  end

  def age_item(item)
    item.sell_in -= 1
  end
end

class Item
  attr_accessor :name, :sell_in, :quality

  def initialize(name, sell_in, quality)
    @name = name
    @sell_in = sell_in
    @quality = quality
  end

  def to_s
    "#{@name}, #{@sell_in}, #{@quality}"
  end
end

So the whole kata took about an hour. We ended up with far more readable code, that makes it easy to expand with more business logic and handles all previous cases and a new one.

To add more "edge cases" we simply pattern match them, use the methods to either set the quality to bypass the max and minimum validations of quality and always use the same age method if needed. Any fundamental changes would be easy to make too, like changing the way things age (just modify the simplistic method, that already has been used in all needed cases).

What about AI?

You might be scared of the incoming singularity, that will assimilate the human race under the new management of the machines. Google, Meta and some other companies are in a race to achieve that, since it makes their lines go up on some business charts. Seemingly forgetting that once they make AI replace everyone at their jobs, there won't be too many people to afford their products, no people to take data from to sell.

Some programmers are afraid of it too. What if some nasty client with a bad codebase, just feeds this code to AI and asks it to refactor it?

This is what awaits us... but not just yet

As a test I did that. I used GitHub Copilot, ChatGPT and if you used those tools for trying to refactor some code, you probably know what they did.

They just moved the strings and magic numbers to CONSTANTS and moved almost every single line into a separate method to just name the actions.

I will not try to explain how little value does this provide, and why is it inferior to the existing code, cause it would be a waste of everyone's time.

Conclusion

Given the "good-looks" nature of Ruby, you could even show the two versions of the code to a non-technical client. Anyone with eyesight would see a total mess in the nested ifs version, and the relative harmony in the pattern match, where each line seems to be looking quite similar. Same goes for RSpec, where the contexts and it descriptions seem to describe the entire domain, that you can verify with the product owner/whoever, showing them the value of it, when the descriptive text actually validates the inner workings of the code.

In the end, if you use the word money a lot in those difficult conversations, you should get some more trust from the people you talk to, cause they like to know, they are working with people, not programming machines. That is something AI will never give them, and a feature that makes flesh-and-blood programmers still relevant in the world racing to singularity. We can map the business requirements to both code and real life. We can figure out ways to save money, and it just so happens that saving money usually comes down to having a good technology, that is achieved through iteration and following best practices, like testing in case of Ruby.

Hanami's place in the world of Rails

Krzysztof — Mon, 20 May 2024 16:08:20 +0000

Background and motivation for this
AppSignal article and frameworks introductions
- Structure and Architecture of Rails and Hanami
- Ecosystem and Community
- Ease of Use, Adoption, Governance, and Learning Curve
- Performance: Ruby on Rails vs. Hanami
- Testing
- Deployment
- Article conclusion comments
Hanami target audience
Real Hanami vs Rails
Business side of it
Conclusion

Background and motivation

Not so long ago, I've seen an article, featured on (I think) RubyWeekly. It came from AppSignal, a company whose products I respect, and the title immediately grabbed my attention due to the fact it was about Hanami. A subject that has lately been close to me.

I've written some articles about it, programmed some side project in it in the last 6 months. I am also actively trying to contribute to community on bug hunt, documentation, trying to answer some discourse questions or post my own, when they seem validated. In general I really want to help hanami contributors out, cause I really appreciate and value their work. Open source is a hard piece of bread and doing it well is impossible. You can never please everyone when you get paid, so doing it for free is ever more of a delusional idea. Open source projects always make some sacrifices, either in their quality, or the developers personal life. It's a hard job, and I respect it. Specially when it results in admirable software, that I think Hanami is.

To go back to the article - it was not to my liking and I think it only scratched the surface and with some statements, missed some crucial information. It was also very short for a rather big subject. It is a 5 minute read, maybe less. 5 minute to decide on what web framework to use when starting a new project/job? Might be okey when working with pet projects, but not professionally. You need more of a deep dive. I am here to provide a bit more context, expand or correct, and give you my own view.

So please, bear with me, cause this article is a bit longer, but I think it's worth it. I will try to give you a better insight into Hanami, and why you should consider it for your next project.

There is also Table of Contents, as I do not expect you to take it all in one go.

AppSignal article and introductions

As a starter I want to make sure I am not taken the wrong way. This is not an "attack" on the article or its author. I still think it is a valuable write-up by Aestimo Kirina and he did an overall good job with it. But I just think there is more to the subject. Much more, and given the work put in into both Rails and Hanami, authors and contributors to both, deserve a more thorough analysis. They have put a lot of thought into the design of their frameworks, and that is not explored well enough in the article to stay fair to the work put into this software.

Article presents 6 main parts, and a quick introduction to both framework, with some closing thoughts at the end. I will follow this structure and expand on all.

As for the introduction to both frameworks. We all know what Rails is. At the same time, our definitions and understandings of it may differ, sometimes even vastly. There was a time where I thought that none truly follow the pure Rails-way. Writing big models and skinny controllers (or the other way around by some), with callbacks, expansive helpers and logic in the views, split across dozens of partials, just seemed impossible in a big, complex project. But sure enough I have seen projects like this, worked on them, rewrote them, spoke with people that write like this and were happy about it. They were getting paid, often handsomely, and their clients/employees were happy with them.

So I thought "am I the classic example of an overthinking, overengineering developer?".

Maybe I am, but then again, I mostly met people who don't write Rails like this. They follow DDD, they use design patterns they picked up from schools, articles, books, other projects, other languages and frameworks. I've worked on about 20 projects professionally to date, add some minor consulting, code review, open source. I've seen a lot of approaches to Rails, and most of them were valid.

That is the thing with Rails. It is easy. It really is. Ruby is easy. And there is nothing wrong with that.

Because both the language and the framework are easy, it is also easy to modify them, bend them to your needs or likings. So many different apps are built with Rails, but even when you find two that are really similar in their business approach, they might be completely different in how they are built from technological perspective. I do not see this being said too much over the internet, but it really is the case. Rails is flexible. People keep repeating that Rails is/has:

Dry
Convention over configuration
Big community

And so on so on, you know the stick. But this is all you can read in the documentation or any basic tutorial. What the reality is, is that Rails is hugely popular. It has been adopted by multiple programmers and companies. They each brought their own knowledge, ideas and experience to Rails and bend it to their needs. This is why Rails can mean so many different things to different people. I often make assumptions about RoR development based on my approach and then become surprised when I meet someone who has no idea what I am talking about.

This is what I am missing when people compare Rails to other frameworks. RoR is never the same one thing. I doubt any framework is.

But people like to make assumptions. They like to simplify things so that they don't question their own knowledge. They like to be right. And they like to be right fast. So they make assumptions. And they are often wrong.

Some devs liked to make fun of RoR for not scaling or being slow, and just ignore the fact that systems like GitHub or Shopify or AirBnB exists and use RoR.

So what are the Hanami assumptions? Taken from the AppSignal article would be:

small memory footprint
focus on modularity
fast and nimble

In general, they are quite true. Hanami is fast and it is modular and composable. It sure is fast (to run).

But it is a quick introduction the author made, later on delving into some details to drive the same points home. At the end of this article I will present my own understanding of what Hanami is, and if you read the whole thing I really hope and think, you will see my main point clearly.

After all, you are a programmer. You are very smart.

Structure and Architecture of Rails and Hanami

The author of the article makes a few seemingly minor mistakes in presenting the structure of hanami. The regular Rails structure that is given is obviously correct, but he makes an assumption about Hanami.

As of now, Hanami 2.1 has no persistence layer. Main page has a guide how to add it though, using ROM-rb. Also the new (added in 2.1) view layer is optional.

First of, you could say the same thing about Rails, well at least the view layer being optional in API apps. I would assume there aren't many RoR apps without persistence. You can set up a new rails app without ActiveModel, but by default it is built in. Hanami has it the other way around. You want persistence? Add it yourself. It is planned to be added in 2.2 but for now, you are on your own.

What does that really mean and why is it a big deal?

Remember what I said about Rails? How pretty much every programmer can have a different approach? Convention over configuration is a mantra for Rails, but people forget, that there are different conventions. Adding persistence layer is a big thing, a lot of configuration and it has to follow a certain convention, right? Right now there is no convention and configuration is up to you. Hanami does that a lot. It is a big difference between the two frameworks. Hanami will have you configuring stuff more, and you can't count on existing conventions as much as in Rails, which has a gem for almost everything. Everything has been tried in Rails, so someone, somewhere presented their convention about implementing certain feature/tech in Rails, you can steal it/let it inspire it. In Hanami, you are sort of on your own (unless you try the discourse forum).

In general Rails does not really enforce too much, hanami does. In this case hanami enforces you to configure stuff on your own. We will explore more things it does enforce later on.

But for now lets think more about the structure and architecture. If you follow the other article, or know anything about RoR or Hanami, you know that both use the MVC pattern. But with Rails, it mostly comes down to "fat models, skinny controllers", so controllers end up delegating actions to models and some custom objects, like service objects and what not. I would call this flat MVC. It is surface level. It takes all the letters from MVC, assigns them one object/thing and calls it a day. There goes your convention, now build us a new facebook.

What Hanami offers is full MVC, with far more collaborators to assign responsibilities: real view objects, slices, providers, initializable actions. This is an important thing it enforces. A more thorough, thought-out separation of responsibilities in the MVC pattern. Controller does not mean one file that does all routes for a given scope. It means separate actions, with possibly quite different responsibilities, implementations, scope of actions, routing rules, authorization and so on. Model means what you make of it, but if you follow the recommended approach with ROM it means full on repository patterns, with entities, relations, commands, validations, mappers separation with multiple adapters.
And finally views means actual view objects, much faster than partials, with clear exposures, inputs, outputs, that are composable, DRY, can have different contexts, are easily rendered outside of actions, have clear dependencies etc.

Hanami expands on MVC greatly compared to Rails, yet it remains flexible as much as Rails is, if not more. That is truly admirable to me.

This is the something I was missing in the AppSignal article and that bothered me deeply. MVC pattern is just scratched by Rails, and carefully adapted by Hanami. Both approaches are fine. You build stuff faster in Rails, so a bare-bones approach makes sense. You build stuff slowly and methodically in Hanami, so a more thorough approach makes sense.

Ecosystem and Community

Author mentions that Hanami is relatively new. This is somewhat true, although under the name of Lotus, Hanami has been around for around 9 years. However it went through many changes, and recently the original creator, has departed from the core team and is no longer working on Hanami. This might be seen as a big blow to the community and framework itself, but seeing the wonderful work and efforts of Tim Riley I am quite hopeful for the future of Hanami.
Anyway my main problem with this point in the article is the documentation mention. I often do not like rails documentation. It is very vast, but not split up too well. ActiveStorage especially has a tangled mess of documentation. Hanami documentation also has it's own problems, specially with the search and version splitting, but at least it is split up well, and gets straight to the point with the description of different features.

However, the author is right, saying that if you are looking for something specific in Rails ecosystem, a simple google search will probably get you there. The quality of those solutions is your to verify, but it is hard to come up with a problem that does not have some sort of solution in Rails.

AI thinks RoR devs are happy artists. Most of them are happy, but remember that most artists are prostitutes.

There is really no comparison here. RoR has a monopoly on ruby world. This will not change, ever. Hanami will exist in the shadows, but its Lotus tree will cast its own shadow, and in it, its developers will hide from the burning sun that RoR can be. Smaller community, means higher chance of meeting someone and actually connecting. Hanami being a framework for an advanced, mature developer means community has a bigger ratio of people who you can learn from. It means less content, but more catered, and curated by people who actually use it.

In terms of ecosystem... well Rails has so many, many gems. It is ridiculous how many things people have built for it. It is also amazing. Hanami only has a small slice of that, and a lot of gems are outdated, do not work with hanami 2.x. However it is also a system that encourages DIY approach, and some things do not make sense as a separate gem. If you want a bunch of ready solutions, you will not even consider Hanami, not just based on gems or not, but based on its approach to development. Rails is fast. Rails is fast to write and run. Hanami is fast to run. You can write Hanami fast if you master it, sure, but you can also quickly write bad Rails code. Hanami makes it way harder to write bad code, makes it harder to add a gem, that later shows it true colors and breaks, or does not meet expectations, gets in the way, is limited etc. Code that is written fast is always more prone to breaking than the code that was written slowly and methodically.

If you ever used dry-rb gems, Hanami is in a big part based on them. They handle most backend problems in a way that takes the best of both worlds of functional and object oriented programming. They are not easy to learn, but they are very powerful. Some of them have a steep learning curve, but the code you produce with them is just awesome. Hanami is not a framework for beginners. It is a framework for people who want to learn, who want to be better, who want to build better software, but are already mature enough, or spent enough time with other systems, to know they need that change, that they need to make things harder for themselves, to be better and do better work. It is a framework for people who want to be proud of their work. Rails can be that too, but it will sometimes make it more difficult to do your best. Rails makes it easy to take the easy way out, take some shortcuts.

Ease of Use, Adoption, Governance, and Learning Curve

AppSignal article makes a good, quick point about Learning and as I mentioned above, Hanami is not for beginners. It is more more mature programmers, who want to learn. Rails is awesome for beginners and seniors alike.

Industry adoption is also not a real comparison, Rails dominates ruby world, and has been adopted by some huge companies and products. There are also no job postings in hanami. From my scouting in the community it appears that every company using hanami is a product company with a senior developer team, who made the decision themselves. None really looks for a hanami developer.

But all that is very obvious and does not need an article.

Governance on the other hand is a whole different beast. Article shows an example of DHH announcement from January that has half a million views. It is an exciting announcement and it is really nice seeing DHH so motivated. He has build something great along with many awesome people, and his undying dedication is admirable. Rails is however far from having a bus problem. If DHH has a change of heart, or (god forbid) something happens to him, Rails will still be here, alive and well, although sad. It is a huge project, with a lot of people working on it, and a lot of people using it. It is a huge part of the ruby community, and it will not go away.

What about Hanami? As I mentioned Luca left, the DHH of Hanami world has left. For now the project is alive, but there are no big companies using hanami, no GitHub or AirBnB, no Shopify. If in few years, Tim Riley leaves will there be someone to pick up the torch? If you look at the team page of Hanami, it is only a handful of people, and as of late, only Tim is truly active in the community and github issues etc.

This is a guessing game however. Both projects are open source, none can really tell where they will go and I don't want to speculate. I hope both projects thrive and its authors can find balance in maintaining it and having a life.

Performance: Ruby on Rails vs. Hanami

Now, the article shows a simple comparison. Hanami is 3x faster in average latency and handles 3x more request. In fact, hanami speed is really hard to beat in ruby's world. That is really impressive.

And it does not really matter.

My boss had an article that touches on the subject of benchmarks from a business perspective. People try to sell their thing on speed. Not just in ruby world, where people are a bit butt-hurt about programmers from other languages saying that ruby is slow.

I want to expand on what my boss said.

Starting off with an example:
Koenigsegg Jesko Absolut is supposed to be the fastest car in the world. With 1600 horse power, it reaches the speed of 531 km/h (330 mph). Sound barrier is broken at 767 mph. That car is really, really fast, no doubt about it.

Hanami request handling if it was a car

My car is a 2013 Suzuki SX4. It was bumped twice, while I left it in a parking lot. One of those left a somewhat significant scratch mark and a dent on the right side of the car. Second time, someone scratched the area above the right wheel in the back. And this other time, I borrowed the car to my dad on a cold, slippery Polish winter and he bumped it too. It still has a visible scratch mark over the left wheel in the back. None of those was fixed and in general the fastest I drove it was like 150 km/h.

But you know what? In any city, and in any country road or intercity road, I will take my Suzuki over that ugly monstrosity. Chance is, in the city I would be faster at my destination, than the angriest Koenigsegg Jesko Absolut driver, cause no doubt he would have an accident or would not make it on polish sharp, tight turns.

Rails request handling if it was a car

There are cases where I would pick Jesko, but they are sparse.

Why? Faster means better right? Well if you are a half-decent driver, you know that is not true. If you drive around the city, you know that speed is not the most important thing. You want a car that handles well, that is good at taking tight turns, that is easy to park, brakes well, and does not burn a lot of fuel when constantly stopping and starting at lights. Suzuki SX4 is a good car for that, and in 6 years, I only had 1 major repair.

What does it has to do with hanami and rails speed? That speed barely matters. Even if hanami router handles the request 3x faster, if we have a big, ugly query written in ROM, cause we don't know how to use it well, we will present a response slowly to the client. In comparison if we do the same in Rails, which we know, and we know how to use a query on ActiveModel, we will present the response to the client much faster than on hanami. On top of that we will write it much faster. No extra, new repository methods, no separate action and route generation.

Does request speed mean anything to your client? In a sales pitch, maybe. But while you are pressed for time to fix a bug, add new feature? You know the answer to that.

So in general, if ruby is slow, you made it so. Yes there are faster alternatives to Rails and Ruby, but when you see a benchmark comparison, always try to look at a bigger picture. So you know, when someone keeps spouting some benchmarks they made, just...

forgetabutit.jpg)

The speed of the tool you have in your hands is relative to your skill, environment and task you use it for. This is true both for and software. Your car goes max speed of 200mph? Awesome, city limit is 30, and many cities have photo-radars on every streetlights on streets that allow you to hit the gas for more than 1 second.

What's that? You reach 100 km/h in 2 seconds? Nice, the only roads with a road straight enough to do full gas for 2 seconds, have streetlights that turn red when they detect someone speeding. Good luck passing them without hitting someone. My car does not even allow me to break certain speed limits.

If you can't think of how all this translated to software development, you probably haven't been burned by the reality of modern software development or you don't drive in the city. Speed rarely comes first in RoR development. First you make it work, then you refactor it, make it fast and reliable, bug free. If your framework handles request super fast - great, but it won't matter if you write a bad, slow query, or use a bad views structure, write too much logic in it, render too many partials etc. Request handling speed itself won't save you from the responsibility of writing fast code, just like having a fast car, won't make you suddenly take less time to get to work, from one end of the city to another, at rush hour.

And yes, even though rails partials are slow, and hanami-views are much faster, all the points I made above still stand.

Testing

In the article, this is the worst part for me, since author barely touched on the subject. First off, a few things:

Hanami comes with rspec built in (rails does not, it uses MIniTest)
Hanami does not come with persistence layer built in, so factory bot without persistence layer will be useless
If you use ROM, you won't be able to use factory bot, instead there is rom factory
Hanami makes is super easy to test actions as unit specs, since each action is easy to initialize, rails controllers are a pain to initialize, and controller specs are full of magic (and are discouraged nowadays)
Hanami makes it easier to have truly unit tests, with clear, or no dependencies
You can use any testing framework with both frameworks
You can use any testing approach with both frameworks
You can write integration specs or not in both frameworks

What the author misses in this section is that this comparison is useless. Ruby community and RoR community, do not have one approach to testing. There are multiple approaches and tools to use, and both framework have their own defaults, but projects often follow their own rules.

Hanami makes it easier to create a very clear unit specs, that are truly unit specs. Rails makes it a bit harder, cause it has way, way more coupling. Rails has a big global scope, many things are tangled up with models or controllers, and you often cannot test something without them, even if they are, from design perspective, unrelated. If you tried for example rendering a view without controller context, you might know what I mean (using turbo makes it possible, but rails often comes in the way).

In general testing comes down to test framework you choose. RSpec in hanami and RSpec in rails, will result in somewhat similar test suites, depending on how much you customize it for your project. Framework determines what you test, but not exactly how. This depends on you, and Hanami, as I think we established, is better at entrusting more stuff to developers, where Rails relies more on its magic.

Deployment

In short: as long as you can produce a Dockerfile for your project, you have plenty of options for deploying. Hanami does not have any recommendations on deployment, Rails has too many, there is a sea of tutorials on how to deploy rails to (insert the new cloud thing here).

There is no point in dwelling on this one, you know how many options Rails has, and it will only have more with Rails 8 having a planned, default dockerfile and seamless Kamal integration.

There is probably 0 chance of this happening to Hanami, but people are running Hanami on production. You will have to figure your own way, or ask the community to recommend something based on your apps architecture and stack.

Article conclusion comments

Ultimately the article's author summarizes how to choose a framework in 4 sentences and a "Ultimately, what you end up choosing is really up to you" and it is a true statement, but also something you know. There is no right or wrong in choosing a framework. There is only what you know, what you want to learn, what you want to build, what you want to achieve, how you want to do it and... oh, wait, it just goes on? It is more complicated than a short article conclusion?

There are multiple, pretty much endless factors when it comes to choosing how to write new projects.

Over the projects I've worked on, around 6 were greenfield project, 4 of those I was a lead on. Choosing a framework was always obvious - Ruby on Rails. Choosing a tech stack was always a challenge. How do we do frontend? Is StimulusJS enough, or will it be limiting? Do we use tailwind, or do we need custom styling? Vite or importmap? Do we use Devise or do we need a more robust solution, like RodaAuth? Custom CMS, or something ready, but also new, like SpinaCMS?

There are so many questions, and even more answers. You need to plan, but you can't plan endlessly. You always have to ultimately make a decision, and then live with it, master it. I usually steered towards technologies I know well, when time was very limited, and picked new stuff when I felt more safe. Not always though, sometimes I took a risk, and it usually payed off.

What I want you to get from my comments about this article, is to think. "what you end up choosing is really up to you" is very simple and very true, but also remember that if you go around, reading articles about what framework to choose for a new professional project, you are probably not ready to make that decision and lead a project. If you want to base your knowledge about Hanami on a 5-10 minute read, then you are not a mature programmer in my eyes.

You need to try it. You need to take a risk on some small professional project, or sacrifice some free time and code something in it in your free time. There is no other way. Reading documentation and looking at public repos is not gonna cut it. You want knowledge? You want experience and perspective? You gotta earn it yourself. No amount of articles is gonna give it to you. Not this one either. This one wants to debunk some stuff and provide you with a fresh look, maybe a nudge towards a more sceptical and curious approach to choosing a framework and comparing them.

This concludes my thoughts about the article AppSignal posted, that kinda awoken something in me. The rest of this is my own thoughts, and I hope you will find them valuable. They will mostly be about Hanami and my experience with it.

Hanami target audience

Key thing to understand is that Hanami is in no way targeted towards new programmers. Rails is a great first framework, and I am speaking from experience here, since I started with it. Along the way I touched on some pure ruby applications, Sinatra apps, RODA + Sequel stack apps, tried different languages and technologies. Rails remains my main tool at my day job, and I would need a lot of convincing to switch to something else. It is fast, flexible, pleasant to work with most of the time.

Hanami is a bit more complicated. Well, actually it is a lot more complicated. It assumes you, as a programmer, have a baggage of experience. You tried many approaches to code structure of web apps, and you have been burned. Here Hanami offers a return to simplicity. However this simplicity is so simple, that it might become complicated. This probably sounds like a over-bloated, self-loathing statement, or simply makes no sense, but let me explain.

Rails offers a lot of magic. It is a big framework, with a lot of stuff built in. It is easy to write a lot of code in Rails very fast, built big things in Rails in no time. That makes it is easy to write bad code in Rails, it creates many shortcuts, and if you take too many of them, and grew accustomed to them, you are gonna have a bad time.

If you spend a lot of time in rails, and then go write pure ruby, no doubt you will subconsciously use some rails only methods that you assumed were part of ruby STD lib. This is also caused by Rails magic, where things are not explicit, where stuff "just works" and you don't really know how and why. Finding out might take a while in case of Rails, because it is huge, full of callbacks na mysterious, expansive modules, global state etc.

Hanami takes a few steps back, and enforces you to build your own magic if you want it, but at the same time gives you tools to keep this magic well organized and explicit. Components and containers system along with slices, and architecture of configuration and providers are the main tools behind it. Of course this requires you to have experience and knowledge in code design. Without it, you will be lost, you will see no point in it and it will be easier to write just the worse, over-engineered, unreadable code that is impossible to maintain and extend.

But if you get some experience in design, and maybe functional programming (which influence can often be seen in dry-rb libs, that Hanami sits upon), things might start falling into place like blocks of tetris.

I think the main audience that might have their eyes glow while exploring Hanami, are people who saw a lot of bad Rails apps, tried a lot of different things, including functional languages, and always wanted something to take the best parts of all those stuff and mix them into one. It is a dangerous thing, since it makes it easy to create a framework with high bar of entry, and honestly, Hanami kinda does set this bar up high. It entrusts a lot in the programmer, but lets be honest - so does ruby. No static types, no strict rules, no enforced design patterns. Ruby is a language that lets you do whatever you want, and Hanami is a framework that lets you do whatever you want, but also gives you tools to do it well. You just need to learn how to use them correctly.

Real Hanami vs Rails

So what would be my comparison of Rails vs Hanami? To be perfectly honest, going back to the Appsignal article I think it was a bit pointless. No ruby programmer, when starting on a greenfield project, really considers Rails vs Hanami. Maybe some of them, when they get a really small project with a free hand, but then again, at this point this programmer is a senior with experience, and won't need an article to help him choose, he will take a look at the documentation, source, code, try it out etc.

In reality, there might be no comparison between the two. At certain areas, they are very similar, except for using the same language, they make heavy use of Rack, follow the MVC pattern (although with a different approach as we explored earlier). They both give a lot of freedom in terms of design, but Hanami offers more tools aimed at creating the design, Rails does not and in those cases where design matters, Rails does come in the way more often.

But in this manner, we could go on and on and on. Hanami does this, Rails does that, this has certain advantages but, that is more blablabla.

You want a real comparison? Write the same simple app in Hanami and Rails. A CRUD that manages books. We both know in which you will write the CRUD faster. Now add an authorization using email password and OAuth. Also rather clear where you implement it faster, even if you got similar experience in both Rails and hanami.

Now add multitenancy, payment system, add authors, readers, sellers and publishers account. Separate it all into DDD contexts, test it all (ofc I don't expect you to do it, but think about it, use your imagination).

Where do you think is the border, where Rails is no longer faster to develop? Where do you think Hanami starts to shine? It is not at CRUD level, and it might not be at design level if you are an super-uber developer that knows it all, and rides RoR like Tony Hawk does skateboard. But if you are not, you should quickly see which framework makes it easier to enforce a good design. Which one sticks closer to pure ruby and adds less magic. Which ones uses ruby to achieve more with less?

If you prefer simplicity, Hanami is better, but first you need a certain perspective and experience, that will make you see that Hanami makes things simpler rather than more difficult. It also does not makes things faster to develop until a certain point.

That is the real comparison to me. I haven't used the old, stupid mantra of "it depends", and I will not. I don't want to make a list, that will ultimately be pointless, I will not give you bullet points of what rails does better, what hanami does better. They are both tools for programmers, they have been designed a certain way, and developed by other humans who make mistakes and try to correct them. It always falls upon to you to learn those tools and make use of them.

However if you are a Rails programmer you probably often saw those "Rails does not scale", "Rails is slow" etc. articles and rants. They are stupid and tedious, often annoying cause of how ignorant they are. If you want to belittle Hanami (which I saw quite a bit of), be aware you are displaying the same level of ignorance as people bashing Rails for things that are simply not true and come from the lack of knowledge and experience.

Rails and Hanami in business

For the end I leave the nasty part.

There are no jobs in Hanami.

There are no companies that look for Hanami developers. There are companies that use Hanami but they don't really talk about it out load, they don't post job offers for Hanami. They will usually look for Ruby developers most likely.

Asking around in Hanami community about this, I got a few answers. There are a few places in Australia and some other serious apps out there, that do use Hanami, so in general, companies that have their own products, built by experienced programmers, have decided on Hanami. They keep quiet about it, cause why would they not? Senior programmers tend to be protective of their knowledge or simply don't have the time or the will to share it so Hanami lives somewhere in the shadows.

Rails jobs market is still healthy despite the recent IT - and I guess general global - crisis, and companies using Rails will not go away. You still have huge companies using it, and more projects coming in. There is no comparison and it seems the only real way to use Hanami in production is to build your own thing, or work in a product company where you get to choose the tech stack.

Conclusion

This was a bit of a rant, I admit. I wanted to give you a fresh look at Hanami, and I hope I did. I hope you see it as a tool for experienced programmers, who want to build better software, who want to be proud of their work, who want to learn and grow. It is not a tool for beginners, and it is not a tool for people who want to build stuff fast. It is a tool for people who want to build stuff well, and are willing to put in the time and effort to do so.

I see Hanami as a breath of fresh air. It is not perfect, nothing is, I think I made some of it flaws rather clear, and pointed out that not all comparisons were fair. But it is a great tool, also one that I found after trying out many different things, and working with dry-rb for many years. As a mix of functional programming and OOP it is an interesting experiment, one that is very impressive, but a difficult one for sure. It will get stuff wrong and get bugs and mistakes, redesigns. So does Rails and both will continue to do so, as most software does. But both tools are well designed and maintained with passion and care, with no competition between themselves and no real, sensible comparison.

I hope I shed some new perspective on Hanami and its place in Ruby world - in Ruby web world to be precise - dominated and monopolized by a single framework. And if I did not, that's okey, I wanted to ramble anyway. I've been trying out Hanami for over half a year, and so far I learned a lot thanks to it. If I wasted your time, be sure to argue with me on the internet, I promise I will respond.

Hanami Shrine - file handling in Hanami

Krzysztof — Mon, 13 May 2024 15:16:50 +0000

Continuing with the latest streak of Hanami focused posts I am bringing you another example of a common feature and implementation, translated to Hanami.

I recently showed some email-password authentication with Hanami, before that it was progress bar feature, now we will handle image uploads.

As a sidenote, nice thing about writing about Hanami is that you get to use all those beautiful pictures of blossoming trees. This time with a shrine in the background.

Shrine

We will be using shrine and I want to start this post by saying a few words about it.

It is great.

I could end it here, but just to clarify: it is a file attachment toolkit for Ruby applications. It is very flexible and can be used with any ORM, any storage service, and any (relevant) processing library. It is also very well documented and has a lot of plugins. We will be making use of those.

ActiveStorage and Shrine

I worked with ActiveStorage on Rails a lot and I have been frustrated by it many, many times. It is overcomplicated, and does not follow the usual doctrine of Rails, that aims to make developers happy. ActiveStorage does not make me happy. It makes me frustrated. It makes me spend a lot of time on documentation, cause even though I keep using it, due to its design and DSL I cannot, for the life of me, remember multiple important details about it. I need to constantly remind myself how to do certain, even mundane, things with it, despite doing them repeatedly.

Shrine does make me happy. It is stupidly simple. I rarely look up the documentation after working with it on few projects. The documentation is also written well and is very concise, something that a lot of rails guides pages fail at in my opinion.

Hanami with ROM and Shrine

So if you have read my previous posts about Hanami, you probably noticed I am using ROM-RB. As an ORM, rom provides minimum infrastructure for mapping and persistence by design. It presents data with immutable structs. Those structs are disjointed from the layer that persists new data to the database.

You might think "oh-oh", this means we have to go through a lot of hoops to make Shrine work with ROM. But you would be wrong. In fact, you are wrong. Shrine is very flexible and can be used with any ORM, remember? It wasn't always that easy, but with a shrine plugin comes with the gem, but has to be enabled we can make it work, surprisingly easily.

Setup

Once again, we have to start with configuring the tools used. If you have seen previous posts, you can figure out that we need to add a new file in config/providers

#config/providers/shrine.rb
Hanami.app.register_provider(:shrine) do
  prepare do
    require "shrine"
    require "shrine/storage/file_system"
    require "shrine/storage/s3"
  end

  start do
    s3_options = {
      bucket: target["settings"].s3_bucket,
      region: target["settings"].s3_region,
      access_key_id: target["settings"].s3_access_key_id,
      secret_access_key: target["settings"].s3_secret_access_key
    }

    permanent_storage = if Hanami.env?(:test)
                          Shrine::Storage::FileSystem.new("spec/tmp/", prefix: "uploads")
                        else
                          Shrine::Storage::S3.new(**s3_options)
                        end

    Shrine.storages = {
      cache: Shrine::Storage::FileSystem.new("public", prefix: "uploads/cache"), # temporary
      store: permanent_storage
    }

    Shrine.plugin :entity
    Shrine.plugin :cached_attachment_data
    Shrine.plugin :restore_cached_data
    Shrine.plugin :form_assign
    Shrine.plugin :rack_file
    Shrine.plugin :validation_helpers
    Shrine.plugin :determine_mime_type

    register :shrine, Shrine
  end
end

We use quite some plugins, some are for caching the file, handling file through form, validations, mime_types. The most important one is entity which is a plugin for handling files being attaches to immutable objects.
This is a simple setup, that saves the files we add in specs to spec/tmp folder, and real files to s3. It also adds some plugins that we will use in the feature. We also need to add some settings:

#config/settings.rb
module Libus
  class Settings < Hanami::Settings
    setting :s3_bucket, constructor: Types::String
    setting :s3_region, constructor: Types::String
    setting :s3_access_key_id, constructor: Types::String
    setting :s3_secret_access_key, constructor: Types::String
  end
end

As per usual, we need a corresponding .env file with key-value pairs like S3_BUCKET=your_bucket_name etc.

As for the database structure, we need a table that needs a file attachment. Lets say we have a table books. In the migration (assuming it was the initial migration that created the table) we have:

column :image_data, :jsonb, null: true

And other columns. JSONB can also be text, but this setup is explained well in shrinerb. Compare this to active storage that adds two entire tables to your database, and stores all files there. Here you have data that you need, connected to relevant table.

Last part of the setup would be to create our Uploader.

#lib/libus/image_uploader.rb
require 'shrine'

module Libus
  class ImageUploader < Hanami.app["shrine"]
    TYPES = %w[image/jpeg image/png image/webp].freeze
    EXTENSIONS = %w[jpg jpeg png webp].freeze
    MAX_SIZE = 5 * 1024 * 1024
    MIN_SIZE = 1024
    Attacher.validate do
      validate_size MIN_SIZE..MAX_SIZE # 1kB..5MB
      validate_mime_type ImageUploader::TYPES
    end
  end
end

This is a class that will handle... uploading of our files. It describes what kind of files we accept, what is the size limit, and what is the minimum size. It is a simple example, but you can probably already see how you can expand it to fit your needs.

And just as a formality, we need some routes:

#slices/main/config/routes.rb
get '/books', to: 'books.index'
get '/books/:id/edit', to: 'books.edit'
patch '/books/:id', to: 'books.update'

As in the progress bar example, final feature will use a little bit of htmx, so bear in mind that this is also setup and affects the html and routing design.

Hanami code

So we have a spot in database, we have shrine setup. How do we implement it all? This framework is not described in shrine getting started guide, so we need to start from the ground up. I add some specs:

#spec/requests/image_upload_spec.rb
RSpec.describe 'ImageUploadSpec', :db, type: :request do
  context 'when photo is uploaded' do
    context "when user is logged in" do
      let!(:user) { factory[:user, name: "Guy", email: "my@guy.com"] }
      let!(:book) { factory[:book] }

      it 'changes the photo' do
        login_as user

        patch "/books/#{book.id}", { id: book.id, book: { image: Rack::Test::UploadedFile.new("spec/fixtures/image.png") } }

        expect(JSON.parse(rom.relations[:books].first.image_data)["id"]).to be_a(String)
        expect(last_response.status).to be(302)
      end
    end
  end
end

Very simple happy path test that checks that if we upload an image, it is saved in the database. It uses the simplest way to upload the file, and a simplest way to check the database. If image_data has an id in its jsonb structure, then it has a file attached.

Since we will have to modify our database, we probably will need to use a repository. It does not have a lot right now (only presenting relevant parts):

#slices/main/repositories/books.rb
module Main
  module Repositories
    class Books < Main::Repo[:books]
      struct_namespace Main::Entities
      commands :create, update: :by_pk, delete: :by_pk

      [...]
    end
  end
end

We do have an update command plugged in, but we can't simply update the image_data field it would skip all Shrine validations and checks, basically making the uploader obsolete.
Speaking of uploader, we need to add it to the entity, that was specified in the repository:

# slices/main/entities/book.rb
module Main
  module Entities
    class Book < ROM::Struct
      include Libus::ImageUploader::Attachment(:image)
      attr_writer :image_data
    end
  end
end

attr_writer is needed cause normally we don't change stuff on entities, but in this case, we need to change the image_data field, so we need to add a writer for it.
Of course that does not mean that the entity will gain the ability to change the data, it is still not its responsibility. But in case we use entity in a form, we can change the data in the form, and then pass it to the repository.

As for the include, it means that the Uploader will do its work on image related fields, meaning it expects image_data attribute on the connected object instance. If we provide that, we get access to all the shrine goods.

We should deal with the repository side and check if attaching the image does its job:

#spec/slices/main/repositories/books_spec.rb
context "#image_attach" do
let!(:dune) { factory[:book, title: "Dune", isbn_13: "9780441172719"] }
  it "succeeds" do
    expect(described_class.new.image_attach(
      dune.id,
      Rack::Test::UploadedFile.new("spec/fixtures/image.png")
    ).image).to be_a(Libus::ImageUploader::UploadedFile)
  end
end

A bit more precise spec, that assumes we will add a new method to the repo, that will attach an image and that the returned entity will have access to the image method entity and that also assumes that entity will have an image method (we did not define it, that is the shrine include doing this work for us).

As for the image_attach method, lets add it to the repository:

#slices/main/repositories/books.rb
def image_attach(id, image)
  book = books.by_pk(id).one!
  attacher = book.image_attacher
  attacher.form_assign({"image" => image})
  attacher.finalize
  self.update(book.id, attacher.column_values)
end

It uses the attacher we get access to from the entity, thanks to the include. Attacker gives us form_assign method that handles - you guessed it - uploads coming in form a form. We could use other methods, but for now we only expect files coming in from a form. After that we tell the attacher to finalize the upload, so move the file from cache, to persistent storage. After this we get access to column_values, which is what we should update the database record with.

Step by step, each easier than the last one, we receive the file (in cache), save it to the database, and the permanent storage (S3). Zero magic and only one column needed. 5 lines of code in the repo. We also have a way to check if the file is attached to the entity, and we can use it in the view.

View

Okey, so we got an index view, with list of books. On it, we can click edit on a row, and change the books picture (cover or whatever).
Part of that template could be:

#slices/main/templates/books/index.html.erb
<% books.each do |book| %>
  <tr>
    <th>
      <label>
        <input type="checkbox" class="checkbox" />
      </label>
    </th>
    <td>
      <div id=<%="picture-#{book.id}" %> class="flex">
        <div class="avatar">
          <div class="mask mask-squircle w-32 h-32">
            <%= book.avatar(:sm)%>
          </div>
        </div>
      </div>
    </td>
    <td>
      <div class="font-bold"><%= book.title %></div>
    </td>
    <td>
      <div class="font-bold"><%= book.category %></div>
    </td>
    <th>
      <div class="text-sm opacity-50"><%= book.author.name %></div>
    </th>
    <th>
      <button
        class="btn btn-sm btn-ghost"
        hx-get="/books/<%= book.id %>/edit"
        hx-target="<%="#picture-#{book.id}" %>"
        hx-swap="innerHTML"
        hx-trigger="click">
        Edit
      </button>
    </th>
  </tr>
<% end %>

A quick rundown of what htmx does here: it replaces the content (inner) of the div with the id picture-#{book.id} with the content of the response from the server, that is the edit form. It happens on button click.

Template with the form is nothing special

#slices/main/templates/books/edit.html.erb
<%= form_for :book, "/books/#{id}", method: :patch do |f| %>
  <%= f.file_field :image, hidden: true, value: book.image_data %>
  <%= f.file_field :image %>
  <button class="btn btn-primary" type="submit">Submit</button>
<% end %>

Here you can see why we need image_data attribute in the entity. We use it in the form, so it needs access to it.

Update

At this point, we basically need one line of code to upload the file.

#slices/main/actions/books/update.rb
module Main
  module Actions
    module Books
      class Update < Main::Action
        include Deps[books_repo: 'repositories.books']
        params do
          required(:id).filled(:integer)
          required(:book).schema do
            optional(:image).value(:hash)
          end
        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?

          books_repo.image_attach(request.params[:id], request.params[:book][:image][:tempfile])

          response.redirect_to "/books"
        end
      end
    end
  end
end

image_attach handles the shrine side of things that will validate the file against the rules we have set up. I skipped error handling for this, since it is no different that any other handling, specially since shrine also stores the initially uploaded file in the cache, and inserts it back in the form. Well it does not do so magically, we added the hidden field, that takes its value from image_data attribute.

Conclusion

ActiveStorage needs two tables. It breaks the usual style of rails, where every table is a model clearly defined in our app. Both tables have their corresponding objects, but they are hidden from out regular code, and not really changeable by us. We need to create an association to every model, and use the same tables for files on every model. The implementation is hidden from us, and file validation is not build in. Documentation is weird and the DSL is confusing.

On the other side, we get Hanami, with very clear and explicit way of setting providers and extensions, along with composable Shrine, that only needs one column and an include statement in an entity. Gives us a clear documentation and great DSL.

Hanami and Shrine are a great fit, and this integration was so easy and natural that I really appreciated how far a good design takes you in software development. Sure it might be obvious that good design and in general good code, provide good results, but it is rather rare to see two things integrate so well and seamless. Even if two pieces of software are well written and designed, there might be glitches and a lot of friction cause the design principles might differ. Not here though.

Also please do not take my criticism of ActiveStorage as a general slander of Rails or its contributors. I love Rails, and I owe my career to it and its many great contributors, I still love working with Rails. I just think that ActiveStorage is a bit of a misstep in the otherwise great framework. I am sure it will get better, but for now, I am sticking with Shrine.

Warden of Hanami - hanami.rb basic authentication

Krzysztof — Tue, 07 May 2024 09:06:01 +0000

Well hello there! I've recently been doing more and more in Hanami and wanted to share some of my experiences and thoughts about it. So here I am with a blog post about authentication in Hanami (made on version 2.1.1).

Current State of Authentication in Hanami and Rails

There is no Hanami specific authentication library. Rails has a plethora of solutions, but nothing was created for Hanami (at least for the current version). There are framework agnostic tools though. OAuth solutions are like that, JWT, libraries like Rodauth.
The last one is particularly interesting for Hanami since it is very much in the same "spirit" in terms of design and it is also the most advanced solution on the ruby market.

However for smaller apps it might be an overkill. In "real-life" production systems, overengineering is one of the biggest crimes. This is true any framework and technology, so in Rails you might want to use Rodauth since it is big and interesting and challenging, but then again, if you are building a simple greenfield MVP you do not have the time or need, for a big, complex solution. In those cases Rails developers usually go for Devise. It is one of the most known Rails gems, in multiple Rails surveys it was both number 1 in popularity, likability and "most frustrating" rankings.

It is no surprise, since the gem makes some things really, really simple, but some other things... well let's just say it can get in the way. This is the thing with libs that do a lot. There comes a time where they stop helping, and the time saved, dwindles away when trying to bypass their limitations.

Devise is build on top of another library and if you ever had to customize Devise, you probably saw that underlying lib. It is called Warden

Warden

Warden is a general Rack authentication framework that provides a mechanism to hook into any Rack application and handle authentication. It is a simple and flexible framework that allows you to use a wide range of authentication strategies in your applications.

If the times of last updates to the repo worry you, rest assured. It is well maintained, and there is no need for constant updates. It is a mature library that does not need to be updated every week. It is an interesting case for conversation about maintainability of open source. Often we are taken back by the timestamps of last updates on certain gems, libs. We rarely think that they are not getting updated cause there is no need for it. But that is a big topic, for another time.

Right now just keep in mind that Devise depends on Warden so as long as it does, and Devise is maintained, Warden will be too.

We will be using it, since it is a low level lib, requires us to do a lot of things ourselves. Something that Hanami encourages a lot (and enforces). It gives us just enough to boost our work on authentication features, but does not hold us back, by giving us too many things we do not need, does not hide implementations and is very explicit. So it is very inline with Hanami philosophy and design.

Our task

We will be adding a simple authentication (email + password) to a basic Hanami application. This will include handling secure password storage, registering, logging in.

Existing setup

We have a Hanami 2.1 application ready, with persistence added (ROM-RB) and a frontend with daisyUI (TailwindCSS extension). If you want more links to those, check out my previous post about Hanami. In general a new Hanami 2.1 app is enough, as long as you got persistence in it.

Adding Warden

After adding it to our gemfile and running bundle install (I'll skip those for brevity), we need to add few pieces of setup. First we go to our config

# config/app.rb
require "hanami"
require "warden"

 module Libus
   class App < Hanami::App
     config.actions.sessions = :cookie, {
       key: "libus.session",
       secret: settings.session_secret,
       expire_after: 60*60*24*365
     }
     config.middleware.use Warden::Manager do |manager|
       manager.default_strategies :password
       manager.failure_app =
         lambda do |env|
           Libus::Actions::AuthFailure::Show.new.call(env)
         end
     end
   end
 end

Also add a line to settings

# config/settings.rb
setting :session_secret, constructor: Types::String

First thing we do in settings is enabling sessions, those are disabled in Hanami by default. Enabling them requires generating a random key. You can use SecureRandom.base64(64) to generate one quickly, then put in the .env file under SESSION_SECRET key. settings.rb will read it from there and app.rb takes it from settings. Simple and very clear, something that I really love about Hanami is how it handles setting things up, like plugins, extensions, gems etc.

Another thing we added here is a failure_app which is a key concept in Warden. Devise does that part for you if you use it, so it is likely you never had to delve into it. Here we need to handle it ourselves, and we will do it by having a simple Hanami Action. This will handle failures on authentication. It requires a special app so it can do whatever you desire with a failure. Usually it is just a redirect back, but maybe some extra logic is required, like tracking attempts. If so, then the failure app is a place to use for it.

But before we implement our failure app, we should register Warden as a provider and add our first strategy. After all we cant test out failure if we do not have a system to log in.

# config/providers/warden.rb
Hanami.app.register_provider(:warden) do
  prepare do
    require "bcrypt"
    require "warden"
  end

  start do
    target.start(:persistence)
    Warden::Strategies.add(:password) do
      def valid?
        params['email'] || params['password']
      end

      def authenticate!
        user_repo = Main::Repositories::Users.new(Hanami.app["persistence.rom"])
        user = user_repo.by_email(params["email"])

        if user && user.password_hash == BCrypt::Engine.hash_secret(request.params["password"], user.password_salt)
          return success!(user)
        end
        fail!("Could not log in")
      end
    end

    Warden::Manager.serialize_into_session{|user| user.id }
    Warden::Manager.serialize_from_session{|id| Main::Repositories::Users.new(Hanami.app["persistence.rom"]).by_id(id) }
  end
end

This sets us up with a second most important part of Warden setup. Strategies. We are using a simple password strategy here. We check if the username and password are present, and if they are, we try to authenticate the user. If we fail, we have the failure app kicking in. If we succeed, we return the user, save his data to the session to later reuse.

This is also something that Devise does for you. It has a lot of strategies built in, and you can still add your own on top of it. Here we are doing it ourselves, it really is not that much work to get it working and we avoid a lot of middleware from Devise

Failure App

The most basic way to handle a failed attempt to enter a forbidden place is to show a simple message. Usually most apps just redirect to login page. For now we will just display a simple message for brevity, but it is a regular Hanami Action, so you can redirect here or do whatever, just showcasing that there is no magic here.

module Libus
  module Actions
    module AuthFailure
      class Show < Main::Action
        def handle(request, response)
          response.body = "STRANGER DANGER"
          response.status = 401
        end
      end
    end
  end
end

An important distinction is that the failure app is not the same as handling, for example, a missing current_user, on an endpoint that requires it. It is responsible for handling failed logging only, not for user trying to access an endpoint that requires a signed in user.

Database

We need to setup a Users table. Lets run a migration for that.
rake db:create_migration[create_users]

ROM::SQL.migration do
  change do
    create_table :users do
      primary_key :id
      column :name, String, null: false
      column :email, String, null: false
      column :password_hash, String, null: false
      column :password_salt, String, null: false
    end
  end
end

Easy as that. We store a password hash and salt, we need Bcrypt in our app to ensure our password take bazillion years to crack if someone attempts to do it.

Lets setup our User relation and repository.

# lib/libus/persistence/relations/users.rb
module Libus
  module Persistence
    module Relations
      class Users < ROM::Relation[:sql]
        schema(:users, infer: true) do
        end
      end
    end
  end
end

# lib/libus/persistence/relations/users.rb
module Main
  module Repositories
    class Users < Main::Repo[:users]
      commands :create, update: :by_pk, delete: :by_pk

      def query(conditions)
        users.where(conditions)
      end

      def email_taken?(email)
        users.exist?(email: email)
      end

      def by_id(id)
        users.by_pk(id).one!
      end
    end
  end
end

Simple relations and repository is all we need to get started.

Specs

Would be nice to test all our authentication features, so lets handle setup for using Warden in our specs.

# spec/support/requests.rb
RSpec.shared_context "Rack::Test" do
   # Define the app for Rack::Test requests
  let(:app) do
    Hanami.boot
    Hanami.app
  end
 end

 RSpec.configure do |config|
   config.include Rack::Test::Methods, type: :request
   config.include_context "Rack::Test", type: :request
   config.include Warden::Test::Helpers, type: :request
   config.after(:each, type: :request) { Warden.test_reset! } 
end

# spec/requests/auth_spec.rb
RSpec.describe 'AuthenticationSpec', :db, type: :request do
  context 'when action inherits from authenticated action' do
    context "when user is logged in" do
      let!(:user) { factory[:user, name: "Guy", email: "my@guy.com"] }

      it 'succeeds' do
        login_as user

        get "/search/isbn", { isbn: "978-0-306-40615-7" }

        expect(last_response.status).to be(200)
      end
    end

    context "when there is no user" do
      it "rejects the request, redirects" do
        get "/search/isbn", { isbn: "978-0-306-40615-7" }

        expect(last_response.status).to be(302)
      end
    end
  end
end

Quite simple. The '/search/isbn' request is gonna be validated for user presence, if it fails (user is missing) we will get redirected. For now this test is red since we don't have any routes available only for current users.

Routes

We won't need much, this is how I structured my routes (only showing the relevant ones)

# New user registration
get '/register', to: 'register.new'
post "/users", to: "users.create"

# Session management
get '/login', to: 'login.new'
post "/sessions", to: "sessions.create"
delete "/logout", to: "sessions.destroy"

Actions

Lets start with a basic registration action and spec for it:

RSpec.describe Main::Actions::Users::Create, :db do
  it "works with the right params" do
    params = {
      email: "some@email.com",
      name: "John Doe",
      password: "password",
      password_confirmation: "password"
    }

    response = subject.call(params)
    expect(response).to be_successful
  end

  context "with bad params" do
    let(:params) { Hash[] }
    it "fails with missing params" do
      response = subject.call({})
      expect(response).not_to be_successful
    end
  end

  context "with password missmatch" do
    let(:params) {
      { email: "good@email.com", name: "John Doe", password: "somepassword", password_confirmation: "differentthing" }
    }

    it "fails with password missmatch" do
      response = subject.call(params)
      expect(response).not_to be_successful
    end
  end

  context "with email already taken" do
    let!(:user) { factory[:user, name: "Guy", email: "my@guy.com"] }
    let(:params) { { email: "my@guy.com", name: "John Doe", password: "password", password_confirmation: "password" } }

    it "fails with" do
      response = subject.call(params)
      expect(response).not_to be_successful
    end
  end
end

Basic tests and here is how we can make them pass.

require 'bcrypt'

module Main
  module Actions
    module Users
      class Create < Main::Action
        include Deps[users_repo: "repositories.users"]

        params do
          required(:email).filled(:string)
          required(:name).filled(:string)
          required(:password).filled(:string)
          required(:password_confirmation).filled(:string)

        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?
          halt 422, { errors: "Password must match the confirmation" }.to_json unless request.params[:password] == request.params[:password_confirmation]
          halt 422, { errors: "This email is already taken" }.to_json if users_repo.email_taken?(request.params[:email])

          password_salt = BCrypt::Engine.generate_salt
          password_hash = BCrypt::Engine.hash_secret(request.params[:password], password_salt)
          users_repo.create(
            name: request.params[:name],
            email: request.params[:email],
            password_hash: password_hash,
            password_salt: password_salt
          )
        end
      end
    end
  end
end

I've put everything in Action object, instead of separating it into validator object and maybe some user service object, since there is very little logic, and it makes it more readable in blog post form. Once this logic expands, we will need a different abstraction layer.

With this, you just got to get yourself a registration view, with the standard email, name, password, password confirmation fields, and you are good to go. You will get a new user in the database.

To this we can add login and retrieving the current user from session.

module Main
  module Actions
    module Sessions
      class Create < Main::Action
        include Deps[users_repo: "repositories.users"]

        params do
          required(:email).filled(:string)
          required(:password).filled(:string)
        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?

          request.env['warden'].authenticate!

          user = users_repo.by_email(request.params[:email])

          if user && user.password_hash == BCrypt::Engine.hash_secret(request.params[:password], user.password_salt)
            request.session[:user_id] = user.id
            response.redirect "/"
          else
            halt 401, "Unauthorized"
          end
        end
      end
    end
  end
end

If you pair this up with a simple login view, you can normally enter email and password and log in into the app.

We can add a parent action for actions that require authentication that will have

before :authenticate_user

private

def authenticate_user(request, response)
  response.redirect_to("/login") unless request.env['warden'].user # request.env['warden'].user is also where you get the current user from. It will be a ROM struct cause of our warden setup from before
end

An examplery action, that will make the previous red spec (the one with redirection) green

module Main
  module Actions
    module IsbnSearch
      class Show < Libus::Actions::AuthenticatedAction
        params do
          required(:isbn).filled(:string)
        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?
          Main::Workers::IsbnSearch.perform_async(request.params[:isbn])
          response.render(view, isbn: request.params[:isbn])
        end
      end
    end
  end
end

Log out

Logging out is the simplest part of it all.

In my navbar I have a bit that goes like this:

<% if warden.user %>
  <li class="justify-center">
    <%= form_for :logout, 'logout', method: :delete do %>
      <button type="submit">Logout</button>
    <% end %>
  </li>
<% else %>
  <li class="justify-center"><a href="/login">Login</a></li>
<% end %>

Which I handle with:

module Main
  module Actions
    module Sessions
      class Destroy < Main::Action
        def handle(request, response)
          request.env['warden'].logout
          response.redirect_to("/")
        end
      end
    end
  end
end

Conclusion

Warden requires very little to start working, and in some parts of the code presented, you might have seen, or thought, that Devise does not really add that much there, other than some convenience. It is not exactly true, since Devise gives us (in Rails) the whole emailing system, views, resetting password, flash messages setting, password handling etc. We often modify a lot of those things and most of them rarely become an actually relevant time saver. Building an email system on your own does not take a lot of time. Modifying an existing one, that you did not wrote, and that has its code hidden in lib source code, can be far more time consuming.

Using Devise has often been a gamble for me. I either save a lot of time, or waste a bit of it. When working on Rails I have no problems using it anyway, since I have also become somewhat proficient in modifying it, but in Hanami, when I have no access to it I can better see what are the drawbacks and benefits of it. Also going on level lower in terms of libs used, gives a better understanding of systems and technologies used (sessions, password storing etc).

I am convinced that every RoR developer should at least once build their own authentication system without Devise since it makes certain things too easy and hides a lot of implementation details, that lead to better understanding of cookies, session and using those stuff, understanding where they are available are where they come from. What really happens when you call current_user and how was it set up? What can we do with data stored in session and what is stored there for the authorization purposes?

The system presented here by me is a skeleton, a starting point, to potentially complex and robust system. Feel free to fork the repo from this point and see how difficult it is to connect emails and better views, flash messages (it really isn't). It does not cost a lot of time, but gives you far better control and understanding of what you actually have in your system, while also cutting your dependencies short (devise uses more than just warden), and making your codebase generally smaller, easier to go through, expand, modify.

Hanami and HTMX - progress bar

Krzysztof — Tue, 07 May 2024 09:03:15 +0000

Hi there! I want to show off a little feature I made using hanami, htmx and a little bit of redis + sidekiq.

It is a popular pattern to show a progress bar when you're doing some longer-running task, like uploading a file, or processing some data. I wanted to show you how to do it in hanami, while making the progress bar fill smoothly, and not just jump from 0 to 100% when the task is done.

The app and the task

I have an app running Hanami 2.1, it has persistence setup with ROM and is using hanami-views for rendering, hanami-assets for providing CSS. Pretty much everything that we need for our task is in a slice called main.

Sidekiq is already configured along with assets, tailwindsCSS.

Our app is going to be a personal library management system, where users can track their books, their placement on a shelf, racks, status of borrowing etc.

The main page has an input for ISBN number. That input sends a GET request to the server, which then fetches the book data from google books API, parses the data to fit our own books relation in ROM, checks if we have it saved in our DB, saves it if not, or just returns the finished status if we already have it.
In the meantime, users sees a progress bar that fills up smoothly, and when the task is done, the progress bar disappears and the book data is shown.

The resulting code will have to use some sleep statements to show the "animation" on the progress bar since normally it is too fast to notice.

Redis

Redis setup is super easy thanks to providers

#config/providers/redis.rb
Hanami.app.register_provider(:redis) do
  prepare do
    require "redis"
  end

  start do
    client ||= ConnectionPool::Wrapper.new do
      Redis.new(url: target["settings"].redis_url)
    end

    register "redis", client
  end
end

#config/settings.rb
setting :redis_url, default: "redis://localhost:6379", constructor: Types::String

Now when we do include Deps['redis] we get a redis client that we can use to store/read our progress data.

HTMX

HTMX has a couple of ways to install. I chose npm because I already used it in the project for tailwindCSS and I have hanami-assets configured for assets.

//slices/main/assets/js/app.js
import "../css/app.css";
import 'htmx.org';
import './htmx.js'

//slices/main/assets/js/htmx.js
window.htmx = require('htmx.org');

And this is all it takes to have HTMX working on your hanami project. If you want to see a quick test example to make sure it works, check out this commit on the demo repo.

Actions and Views

We need few blocks in place: downloading the data, parsing it, saving it to DB, and checking if we already have it. All of this needs to be tracked by the backend and monitored by frontend to show the progress.

So lets start with displaying everything, since we can use placeholders for that.

Our slice will have 3 relevant actions: IsbnSearch, SearchProgress and SearchResult.

    scope 'search' do
      get '/isbn', to: 'isbn_search.show'
      get '/progress', to: 'search_progress.show'
      get '/result', to: 'search_result.show'
    end

First step will be the search:

module Main
  module Actions
    module IsbnSearch
      class Show < Main::Action
        params do
          required(:isbn).filled(:string)
        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?
          Main::Workers::IsbnSearch.perform_async(request.params[:isbn])
          response.render(view, isbn: request.params[:isbn])
        end
      end
    end
  end
end

# Spec, only showing the basic happy path example, for breviety
context "with good params"  do
  let(:params) { Hash[isbn: "978-0-306-40615-7"] }
  let(:worker) { double(Main::Workers::IsbnSearch) }
  it "works" do
    Sidekiq::Testing.fake! do
      response = subject.call(params)
      allow(Main::Workers::IsbnSearch).to receive(:perform_async)
       .with("978-0-306-40615-7")
       .and_return(worker)
      expect(response.status).to eq 200
    end
  end
end

Then we have the view object

module Main
  module Views
    module IsbnSearch
      class Show < Main::View
        config.layout = nil

        expose :isbn do |isbn:|
          { type: isbn.size, identifier: isbn }
        end
      end
    end
  end
end

With the template:

<input hidden id="isbn-type" name="isbn[type]" value="<%= isbn[:type] %>">
<input hidden id="isbn-identifier" 
                       name="isbn[identifier]" 
                       value="<%= isbn[:identifier] %>">

<div
    hx-trigger="done"
    hx-get="/search/result"
    hx-include="#isbn-type, #isbn-identifier"
    hx-swap="outerHTML"
    hx-target="this">
  <h3 role="status" id="pblabel" tabindex="-1" autofocus>Searching</h3>

  <div
    hx-get="/search/progress"
    hx-include="#isbn-type, #isbn-identifier"
    hx-trigger="every 2000ms"
    hx-target="this"
    hx-swap="innerHTML">

    <div class="w-64 h-5 mb-5 overflow-hidden bg-base-100 rounded-md shadow-inner" 
            role="progressbar"
             aria-valuemin="0" 
            aria-valuemax="100"
            aria-valuenow="122"
            aria-labelledby="pblabel">
      <div id="pb" class="float-left h-full text-white text-center bg-accent transition-width duration-2000" 
              style="width:0%"></div>
    </div>
  </div>
</div>

This is where the majority of HTMX magic comes in. We have 3 important div's:

The outer div, that will be replaced with the result of the search, and along with him, all divs inside (that are described below)
The progress div, that every 2 seconds checks the progress of the search, and updates the progress bar -> div below
The initial progress bar div, that is replaced with the progress bar div from the SearchProgress action, the div from point 2 makes the request.

All this happens based on the hx-target, hx-get and hx-swap attributes, hx-trigger is used to repeat the request every 2 seconds on the div that checks the progress, and for completing the process in case of the outer div.

target tells HTMX where to put the response
get is simply the request identifier, the get is the type and the value given is the URL
swap is a method of replacement, so how to replace the target with the response

This is all rather basic stuff from HTMX, pretty much the introduction part of their docs. So if you think it is somewhat impressive, keep it mind this is just what is possible with the very basics. Although I do think that hanami routing and actions and view systems make it much easier to use (than for example rails would). In Hanami it is simply very easy to make the code very modular, composable and repeatable. I think HTMX and Hanami are a great fit together and I hope this whole example does show it.

So lets take a look at the SearchProgress action:

module SearchProgress
  class Show < Main::Action
    include Deps["redis"]
    def handle(request, response)
      if redis.hget("isbn_search", request.params[:isbn][:identifier]).to_i == 3
        response.headers["HX-Trigger"] = "done" 
      end
      response.render(view, isbn: request.params[:isbn])
    end
  end
end

HX-Trigger is a header that will tell the HTMX to trigger the done action, which will fire up the SearchResult action, and that will render the result of the search.
The header is based on the redis value, which is set by the worker (we will get there soon).

module SearchResult
  class Show < Main::Action
    include Deps["repositories.books"]

    def handle(request, response)
      response.render(
        view,
        book_found: books.by_isbn(type: request.params[:isbn][:type].to_i,
        identifier: request.params[:isbn][:identifier])
      )
    end
  end
end

Result is simple, we use the books repo to make a simple request, som rom and sequel are used here, but it is not really relevant to the topic at hand, so the repo implementation is omitted.

You probably noticed the inputs in the view, they just hold the values we need to keep finding the correct book based on the initial isbn (which later got an identifier on the back, that tells us if it is ISBN-10 or 13 to make some queries easier).

The templates are also important, here is the template for progress along with its view object:

<div class="w-64 h-5 mb-5 overflow-hidden bg-gray-300 rounded-md shadow-inner" 
        role="progressbar" 
        aria-valuemin="0" 
        aria-valuemax="100" 
        aria-valuenow="<%= search_progress %>" 
        aria-labelledby="pblabel">
  <div id="pb" class="float-left h-full text-white text-center bg-accent transition-width duration-2000" 
         style="width: <%= search_progress %>%">
  </div>
</div>

module Main
  module Views
    module SearchProgress
      class Show < Main::View
        config.layout = nil
        include Deps["redis"]
        expose :search_progress, decorate: false do |isbn:|
          search_progress(isbn: isbn)
        end

        private

        def search_progress(isbn:)
          progress = redis.hget("isbn_search", isbn[:identifier])
          case progress.to_i
          when 1
            40
          when 2
            80
          when 3
            100
          else
            10
          end
        end
      end
    end
  end
end

Just a simple check of the redis value, and returning the correct value for the progress bar. Then the rest is taken care by CSS and tailwind styling. Thanks to sharing the id between the HTML elements that are "replaced", the transition is smooth, rather than the filler color of the bar just appearing. It flows.

Then we have the result that replaces the progress bar, it is a simple card with the book data, taken from the struct we got from the repository.

<div class="card lg:card-side bg-base-100 shadow-xl">
  <figure class="w-1/2 h-1/2 max-w-96 max-h-96">
    <img src="<%= book.image_url.gsub("http", "https") %>" alt="Cover" />
  </figure>
  <div class="card-body">
    <h2 class="card-title">
      <%= book.title %>
    </h2>
    <p>
      <%= book.description[0..500] %>
    </p>
    <div class="card-actions justify-end">
      <button class="btn btn-primary">See more</button>
    </div>
  </div>
</div>

module SearchResult
  class Show < Main::View
    config.layout = nil

    expose :book, decorate: false do |book_found:|
      book_found
    end
  end
end

Those are all the blocks we need in place to have a fully functional frontend. We just need the backend to implement all the logic that does the actual work, and monitors the progress.

Worker that works hard

For brevity's sake I will only show the basic spec, with the conditions we wanna meet.

We wanna take an ISBN, and delegate work to other objects, checking if the book was saved (this is a scenario where the ISBN is for a new book), and that redis status was updated correctly at the end.

  context 'when called' do
    let(:google_isbn_service) { double(Main::Services::GetGoogleIsbn) }
    let(:parser) { double(Main::Services::GetGoogleIsbn) }
    let(:parse_output) {
      { title: "Dune",
        description: "A nice SF book",
        image_url: "https://www.some.site",
        published_date: "1965",
        category: "SF",
        language: "en",
        authors: ["Frank Herbert"],
        isbn_numbers: [
          { type: 10, identifier: "0441172717" },
          { type: 13, identifier: "9780441172719" }
        ]
      }

    }

    it 'processes the job' do
      expect(Main::Services::GetGoogleIsbn).to receive(:new).and_return(google_isbn_service)
      expect(Main::Parsers::Google::Isbn).to receive(:new).and_return(parser)
      expect(google_isbn_service).to receive(:call).with(isbn: "9780441172719").and_return({ body: "some body" })
      expect(parser).to receive(:parse).with(json: { body: "some body" }).and_return(parse_output)
      expect{Main::Workers::IsbnSearch.perform_async("9780441172719")}.to change{db[:books].count}.by(1)
      expect(Hanami.app["redis"].hget("isbn_search", "9780441172719")).to eq("3")
    end
  end

In general sidekiq workers should only take the most basic input (strings etc.) and delegate most of the work to other objects, so that we can test them in isolation, handle errors better etc. This is the way I always preferred the workers/jobs to be coded, and it works great here too, cause we just list all the dependencies, use them one by one and monitor the process easily and clearly thanks to that.

For brevity's sake I've put everything in the perform method, but it in real life it could be a good idea to split it out more into methods, for readability and clearer error outputs.

module Main
  module Workers
    class IsbnSearch
      include Sidekiq::Job
      include Deps[
                "services.get_google_isbn",
                "redis",
                "persistence.rom",
                "repositories.books",
                parser: "parsers.google.isbn"]

      def perform(isbn)
        redis.hset("isbn_search", { isbn => 1 })

        output = parser.parse(json: get_google_isbn.call(isbn:))

        redis.hset("isbn_search", { isbn => 2 })

        if books.by_isbn(type: 10, identifier: isbn) || books.by_isbn(type: 13, identifier: isbn)
          redis.hset("isbn_search", { isbn => 3 })
        else
          rom.relations[:books].transaction do
            author = rom.relations[:authors].changeset(
              :create, { name: output[:authors].join(', ') }
            ).commit
            new_book = rom.relations[:books].changeset(
              :create, output[:data]
            ).commit
            redis.hset("isbn_search", { isbn => 3 })
          end
        end
      end
    end
  end
end

Now this is a fully functional progress bar (the implementations of parser and get_google_isbn objects are not really relevant, those are implementation details not connected to progress bar directly).

This could be further improved and made into a far more reusable code, with redis communication being delegated to pub/sub system, that could be better coupled with places that do actually push the progress forward, rather than putting everything into a worker, that should be more of a simple delegator.

But maybe pub/sub system in hanami is a topic for a different episode?

This gives us a reactive progress bar, that updates every second and in the end, switches place with the ready content.

You can see a gif of it working on the original article page, if you do not believe me.

Summary

So this is a neat feature using Hanami and HTMX, backed by redis and sidekiq. I hope this was a nice mix of familiar and unfamiliar technologies, to get you interested in something new, like Hanami or HTMX.

Trailblazer tutorial: collections, forms, testing Cells - part 6

Krzysztof — Tue, 19 Nov 2019 10:24:53 +0000

In this post, we will finish up refactoring views that we saw, including a view that is heavily based on callbacks. We won't be able to delete them yet since for now we only render the cell in one view, all the others will still use ActionView and those callbacks, but we will be able to get rid of them for this one place at least. We will also get to know how to use collections for cells and how to write some tests for them.

Let's start by finishing the TODO's we left last time.


- if display_staff_event_subnav?
  = cell(Navigation::Staff::Cell::Event)
- elsif schedule_mode?
  = cell(Navigation::Staff::Cell::Schedule)

As you can see we nest another concept into Navigation, called Staff. Then we create another folder cell and another folder view. This keeps our code structured logically. Keeping to this convention you might have a lot of files called event.rb or event.haml but they will be in their right scope so it will be even easier to find them, depending on the context you want the view for even for. You might have events displayed differently in many places in the app like the navbar event will have a different display that event-specific show view.

This will be represented accordingly in your Trailblazer files structure. In those cells, we will also use inheritance, cause we need methods that are in the Main cell. It is worth mentioning that cells allow us to inherit entire views, not just class methods. You can also inherit ONLY the view, without the class through inherit_views Something::Cell. But by default, you will also get the view inheritance as a "backup" for your view. This means that our Event cell will look for a view in its parent folder scope if the view for its cell is not found.


Navigation::Staff::Cell::Event.prefixes => ["app/concepts/navigation/staff/view", "app/concepts/navigation/view"]

The cells and views are nothing new by themselves for you.

/* app/concepts/navigation/staff/view/schedule.haml */
.navbar.navbar-fixed-top.schedule-subnav
  .container-fluid
    %ul.nav.navbar-nav.navbar-right
    %li{class: subnav_item_class('event-schedule-grid-link')}
        %a{:href => event_staff_schedule_grid_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-calendar-times-o
        %span Grid
    %li{class: subnav_item_class('event-schedule-time-slots-link')}
        %a{:href => event_staff_schedule_time_slots_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-clock-o
        %span Time Slots
    %li{class: subnav_item_class('event-schedule-rooms-link')}
        %a{:href => event_staff_schedule_rooms_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-tag
        %span Rooms


# app/concepts/navigation/staff/cell/schedule.rb
module Navigation
  module Staff
    module Cell
      class Schedule < Navigation::Cell::Main
        alias event model
        property :slug
      end
    end
  end
end

We only change the inheritance, the rest is known. However, we have a problem to deal with - the mentioned callbacks that are going to be called when entering this view:


/* app/concepts/navigation/view/main.haml */
- elsif program_mode?
  = render partial: "layouts/nav/staff/program_subnav"

Let's create the cell and the view, paste the partial content into it and start dealing with callbacks one by one. program_mode? comes from ApplicationController itself. There it bases the result of the method on an instance variable set by a callback from a concern called ProgramSupport. This concern is included in controllers that will want to render this partial.

So, if we will aim to refactor the whole app using cells, we will be able to clean up ApplicationController and get rid of some concern that includes callbacks. A lot of benefits on the horizon. But right now we are limited to fixing one place. We need to define the condition anew. The ProgramSupport also has some helper methods that we will need to maintain the same state of our page.

So this is the concern:


# app/controllers/concerns/program_support.rb
require 'active_support/concern'

module ProgramSupport
  extend ActiveSupport::Concern

  included do
    before_action :require_program_team
    before_action :enable_staff_program_subnav
    before_action :set_proposal_counts

    helper_method :sticky_selected_track
  end

  private

  def sticky_selected_track
    session["event/#{current_event.id}/program/track"] if current_event
  end

  def sticky_selected_track=(id)
    session["event/#{current_event.id}/program/track"] = id if current_event
  end

  def set_proposal_counts
    @all_accepted_count ||= current_event.stats.all_accepted_proposals
    @all_waitlisted_count ||= current_event.stats.all_waitlisted_proposals
    unless sticky_selected_track == 'all'
    @all_accepted_track_count ||= current_event.stats.all_accepted_proposals(sticky_selected_track)
    @all_waitlisted_track_count ||= current_event.stats.all_waitlisted_proposals(sticky_selected_track)
    end
  end
end

And this is how we will remake this into a cell:


# app/concepts/navigation/staff/cell/program.rb
module Navigation
  module Staff
    module Cell
      class Program < Navigation::Cell::Main
        alias event model
        property :slug
        property :stats
        property :tracks

        private

        def program_tracks
          tracks&.any? ? tracks : []
        end

        def sticky_selected_track
          session["event/#{event.id}/program/track"]
        end

        def sticky_selected_track=(id)
          session["event/#{event.id}/program/track"] = id
        end

        def all_accepted_count
          stats.all_accepted_proposals
        end

        def all_waitlisted_count
          stats.all_waitlisted_proposals
        end

        def all_accepted_track_count
          stats.all_accepted_proposals(sticky_selected_track) if sticky_selected_track != 'all'
        end

        def all_waitlisted_track_count
          stats.all_waitlisted_proposals(sticky_selected_track) if sticky_selected_track != 'all'
        end
      end
    end
  end
end

Now we also need to deal with resolving the program_mode? method, which was based on callbacks, being run in certain controllers. The simplest way is that we still have access to the controller being used from the context. So we can simply check from what controller did the cell was rendered from, and if it fits our needs, we will return true in the condition.

No callbacks, no including, no setting variables used in one method. Just a check if the request came from the right place to render the cell. We are still dealing with the navigation bar at the top of the site, so its something displayed with base layouts, so controllers coming into this place will vary. This ends up being out of the view.


/* app/concepts/navigation/staff/view/program.haml */
.navbar.navbar-fixed-top.program-subnav
  .container-fluid
    %ul.nav.navbar-nav.navbar-right
    %li{class: subnav_item_class("event-program-proposals-link")}
        %a{:href => event_staff_program_proposals_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-balance-scale
        %span Proposals
    %li{class: subnav_item_class("event-program-proposals-selection-link")}
        %a{:href => selection_event_staff_program_proposals_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-gavel
        %span Selection
    %li{class: subnav_item_class("event-program-sessions-link")}
        %a{:href => event_staff_program_sessions_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-check
        %span Sessions
    %li{class: subnav_item_class("event-program-speakers-link")}
        %a{:href => event_staff_program_speakers_path(slug), :class => "text-primary", :method => :post}
        %i.fa.fa-users
        %span Speakers

    %ul.nav.navbar-nav.session-counts
    %li.static
        %span.title Sessions:
        %div.counts-container
        .total.all-accepted
            %span Accepted
            %span.badge= all_accepted_count
        .total.all-waitlisted
            %span Waitlisted
            %span.badge= all_waitlisted_count

    - if program_tracks.any?
        %li.static
        %span.title By Track:
        %form.track-select{data: {event: slug}}
            %select{id: 'track-select', name: 'track'}
            %option{value: 'all'} All
            %option{value: ' ', selected: selected?} General
            - program_tracks.each do |track|
                = cell(Navigation::Staff::Cell::TrackRow, track, sticky_selected_track: sticky_selected_track)
        %div.counts-container
            .by-track.all-accepted
            %span Accepted
            %span.badge= all_accepted_track_count
            .by-track.all-waitlisted
            %span Waitlisted
            %span.badge= all_waitlisted_track_count

Notice the nested cell, we pass options there. This is how the sticky_selected_track is later accessible.


# app/concepts/navigation/staff/cell/track_row.rb
module Navigation
  module Staff
    module Cell
      class TrackRow < Trailblazer::Cell
        alias track model

        property :id
        property :name

        private

        def selected?
          id.to_s == options[:sticky_selected_track]
        end
      end
    end
  end
end

/* app/concepts/navigation/staff/view/track_row.haml */
%option{value: id, selected: selected?}= name

And our navigation bar looks complete now, no callbacks in this view. It is also free to be used in the future, so from now on, we would only have to refactor content views (like proposal index that we did), while nav bar stays the same, rendered by our BaseLayout.

Now we can move on to some other options cells give us. Let's start from the collection option and its base usage.


- program_tracks.each do |track|
  = cell(Navigation::Staff::Cell::TrackRow, track, sticky_selected_track: sticky_selected_track)

Consider this code and how many times we've already used each to iterate over the collection and render a cell for each. We can simplify this to:


= cell(Navigation::Staff::Cell::TrackRow, collection: program_tracks, sticky_selected_track: sticky_selected_track)

Now we don't need to pass in the track we simply pass the collection. Each item from it will render a cell with itself as the model of that cell. Going back we can do this for every |each| do we created so far.

- notifications_unread.each do |notification|
  = cell(Navigation::Cell::NotificationRow, notification)

= cell(Navigation::Cell::NotificationRow, collection: notifications_unread)


- invites.each do |invitation|
  = cell(Proposal::Cell::InviteRow, invitation)

= cell(Proposal::Cell::InviteRow, collection: invites)

- talks.each do |talk|
  = cell(Proposal::Cell::TalkRow, talk, event: event)

= cell(Proposal::Cell::TalkRow, collection: talks, event: event)

You can see the comparison and as you can see, we can still pass in options (like the event in the TalkRow). While we are at it, let's check out other options we have with the collection, while not useful for us now, it is worth going over them, to know what we will have access to.

If we ever the cell to use something else than the show method to display its content, we can also call it on the collection by simply changing to:


= cell(Proposal::Cell::TalkRow, collection: talks, event: event).(:method)

Cells from the collection can be invoked differently, and joining them can also have additional options. Each item from collection will have their show method invoked, and then joined using the additional options passes to join like so:


= cell(:something, collection: Something.all).join("<hr/>") do |cell, i|
  i.odd? ? cell.(:show) : cell(:alternate_method)
end
# => "rendered with show\n<hr/>alternate render\n<hr/>rendered with show\n<hr/>alternate render"

join with an argument to be inserted between renders can be also called without a block:


= cell(Proposal::Cell::InviteRow, collection: invites).join("br /")

We should now talk about testing. I know in normal TDD we should have started with that, but this tutorial was focused on building and using cells to refactor the view of an application. So after doing that we should check if our tests are still good, and we can add cell tests to that. Our goal is to get rid of the controller test, and only have unit and integration tests (approach suggested by TB on its main page).

The controller's test still passes.

bundle exec rspec spec/controllers/proposals_controller_spec.rb
...................

Finished in 1.67 seconds (files took 3.29 seconds to load)
19 examples, 0 failures

We need to deal with integration tests though. This will be a lot more work, cause we also have to account for changes made before when creating operations and contracts. We will, however, aim to have still the same integration tests and add tests for cells on top of that. In the long term, this would eventually let us get rid of controllers' tests, along with adding more and more unit tests to operations. For now, we are modifying smaller parts of the app so we only add new stuff and change existing, without deleting things that eventually will become obsolete and unused.

feature 'Proposals' do
  let!(:user) { create(:user) }
  let!(:event) { create(:event, state: 'open') }
  let!(:closed_event) { create(:event, state: 'closed') }
  let!(:session_format) { create(:session_format, name: 'Only format') }
  let(:session_format2) { create(:session_format, name: '2nd format') }

  let(:go_to_new_proposal) { visit new_event_proposal_path(event_slug: event.slug) }

  let(:create_proposal) do
    fill_in 'Title', with: 'General Principles Derived by Magic from My Personal Experience'
    fill_in 'Abstract', with: 'Because certain things happened to me, they will happen in just the same manner to everyone.'
    fill_in 'proposal_speakers_attributes_0_bio', with: 'I am awesome.'
    fill_in 'Pitch', with: 'You live but once; you might as well be amusing. - Coco Chanel'
    fill_in 'Details', with: 'Plans are nothing; planning is everything. - Dwight D. Eisenhower'
    select 'Only format', from: 'Session format'
    click_button 'Submit'
  end

  let(:create_invalid_proposal) do
    fill_in 'proposal_speakers_attributes_0_bio', with: 'I am a great speaker!.'
    fill_in 'Pitch', with: 'You live but once; you might as well be amusing. - Coco Chanel'
    fill_in 'Details', with: 'Plans are nothing; planning is everything. - Dwight D. Eisenhower'
    click_button 'Submit'
  end

  before { login_as(user) }
  after { ActionMailer::Base.deliveries.clear }

  context 'when navigating to new proposal page' do
    context 'after closing time' do
      it 'redirects and displays flash' do
        visit new_event_proposal_path(event_slug: closed_event.slug)

        expect(current_path).to eq event_path(closed_event)
        expect(page).to have_text('The CFP is closed for proposal submissions.')
      end
    end
  end

  context 'when submitting' do
    context 'with invalid proposal' do
      before :each do
        go_to_new_proposal
        create_invalid_proposal
      end

      it 'submits unsuccessfully' do
        expect(page).to have_text('There was a problem saving your proposal.')
      end

      it 'shows Title validation if blank on submit' do
        expect(page).to have_text("Title *can't be blank")
      end

      it 'shows Abstract validation if blank on submit' do
        expect(page).to have_text("Abstract *can't be blank")
      end
    end

    context 'less than one hour after CFP closes' do
      before :each do
        go_to_new_proposal
        event.update(state: 'closed')
        event.update(closes_at: 55.minutes.ago)
        create_proposal
      end

      it 'submits successfully' do
        expect(page).to have_text('Thank you! Your proposal has been submitted and may be reviewed at any time while the CFP is open.')
      end
    end

    context 'more than one hour after CFP closes' do
      before :each do
        go_to_new_proposal
        event.update(state: 'closed')
        event.update(closes_at: 65.minutes.ago)
        create_proposal
      end

      it 'does not submit' do
        expect(page).to have_text('The CFP is closed for proposal submissions.')
      end
    end

    context 'with Session Formats' do
      # Default if one Session Format that it is auto-selected
      it "doesn't show session format validation if one session format" do
        go_to_new_proposal
        create_invalid_proposal
        expect(page).to_not have_text("Session format *None selected Only format 2nd formatcan't be blank")
      end

      it 'shows Session Format validation if two session formats' do
        skip 'Address after session format change'
        session_format2.save!
        go_to_new_proposal
        create_invalid_proposal
        expect(page).to have_text("Session format *None selected Only format 2nd formatcan't be blank")
      end
    end

    context 'with an existing bio' do
      before { user.update_attribute(:name, 'new speaker') }
      before { user.update_attribute(:bio, 'new bio') }

      it "shows the user's bio in the bio field" do
        go_to_new_proposal
        expect(page).to have_field('Bio', with: 'new bio')
      end

      it "shows the user's name in the name field" do
        go_to_new_proposal
        expect(page).to have_field('Name', disabled: true, with: 'new speaker')
      end
    end

    context 'With Pitch and Details' do
      before :each do
        go_to_new_proposal
        create_proposal
      end

      it 'submits successfully' do
        expect(Proposal.last.abstract).to_not match('<p>')
        expect(Proposal.last.abstract).to_not match('</p>')
        expect(page).to have_text('Thank you! Your proposal has been submitted and may be reviewed at any time while the CFP is open.')
      end

      it 'does not create an empty comment' do
        expect(Proposal.last.public_comments).to be_empty
      end

      it "shows the proposal on the user's proposals list" do
        visit proposals_path
        within(:css, 'div.proposals') do
          expect(page).to have_text('General Principles')
        end
      end
    end
  end

  context 'when editing' do
    scenario 'User edits their proposal' do
      go_to_new_proposal
      create_proposal
      click_link 'Edit'

      expect(page).to_not have_text('A new title')
      fill_in 'Title', with: 'A new title'
      click_button 'Submit'
      expect(page).to have_text('A new title')
    end
  end

  context 'when commenting' do
    before :each do
      go_to_new_proposal
      create_proposal
      fill_in 'public_comment_body', with: "Here's a comment for you!"
      click_button 'Comment'
    end

    scenario 'User comments on their proposal' do
      expect(page).to have_text("Here's a comment for you!")
    end

    it "it does not show the speaker's name" do
      within(:css, '.speaker-comment') do
        expect(page).to have_text('speaker')
      end
    end
  end

  context 'when confirming' do
    let(:proposal) { create(:proposal) }

    before { proposal.update(state: Proposal::State::ACCEPTED) }

    context 'when the proposal has not yet been confirmed' do
      let!(:speaker) { create(:speaker, proposal: proposal, user: user) }

      before do
        visit event_proposal_path(event_slug: proposal.event.slug, uuid: proposal)
        click_link 'Confirm'
      end

      it 'marks the proposal as confirmed' do
        expect(proposal.reload.confirmed?).to be_truthy
      end

      it 'redirects the user to the proposal page' do
        expect(current_path).to eq(event_proposal_path(event_slug: proposal.event.slug, uuid: proposal))
        expect(page).to have_text(proposal.title)
        expect(page).to have_text("You have confirmed your participation in #{proposal.event.name}.")
      end
    end

    context 'when the proposal has already been confirmed' do
      let!(:speaker) { create(:speaker, proposal: proposal, user: user) }

      before do
        proposal.update(confirmed_at: DateTime.now)
        visit event_proposal_path(event_slug: proposal.event.slug, uuid: proposal)
      end

      it 'does not show the confirmation link' do
        expect(page).not_to have_link('Confirm my participation')
      end
    end
  end

  context 'when deleted' do
    let(:proposal) { create(:proposal, event: event, state: Proposal::State::SUBMITTED) }
    let!(:speaker) { create(:speaker, proposal: proposal, user: user) }

    before do
      visit event_proposal_path(event_slug: event.slug, uuid: proposal)
      click_link 'delete'
    end

    it 'redirects to the proposal show page' do
      expect(page).not_to have_text(proposal.title)
    end
  end

  context 'when withdrawn' do
    let(:proposal) { create(:proposal, :with_reviewer_public_comment, event: event, state: Proposal::State::SUBMITTED) }
    let!(:speaker) { create(:speaker, proposal: proposal, user: user) }

    before do
      visit event_proposal_path(event_slug: event.slug, uuid: proposal)
      click_link 'Withdraw'
      expect(page).to have_content('As requested, your talk has been removed for consideration.')
    end

    it 'sends a notification to reviewers' do
      expect(Notification.count).to eq(1)
    end

    it 'redirects to the proposal show page' do
      expect(page).to have_text(proposal.title)
    end
  end
end

This is what we are dealing with. The functionalities of the page stayed the same, but now we have operations, contracts, and cells. So we need to maintain those functionalities.

Finished in 9.71 seconds (files took 3.26 seconds to load)
22 examples, 6 failures, 1 pending

The first failure is from Proposals when submitting with invalid proposal submits unsuccessfully context.

Failure/Error: expect(page).to have_text('There was a problem saving your proposal.')
expected to find text "There was a problem saving your proposal." in "Fine Event3 CFP View all notifications User Name 2 My Profile Sign Out Proposal creation failure Fine Event3 Dec 02 - 07, 2019 http://fineevent.com/ Fine Event3 CFP CFP open CFP closes: Nov 28, 2019 at 12:46pm UTC 21 days left to submit your proposal Submit a proposal We want all the good talks! Fine Event3 Contact: Powered by CFP App"

Since not all validations were yet moved from model to contract, and our test provides data required by contract, the test fails on persist step, cause it validates Proposal as ApplicationRecord instance, which has additional validations. This is easily found out, using trace and wtf?.

result = Proposal::Operation::Create.trace(params: params, current_user: current_user)

result.wtf?
`-- Proposal::Operation::Create
   |-- Start.default
   |-- event
   |-- model.build
   |-- assign_event
   |-- contract.build
   |-- event_open?
   |-- contract.default.validate
   |   |-- Start.default
   |   |-- contract.default.params_extract
   |   |-- contract.default.call
   |   `-- End.success
   |-- persist.save
   `-- End.failure

Wee need move those

validates :title, :abstract, :session_format, presence: true

Here:


# app/concepts/proposal/contract/create.rb
module Proposal::Contract
  class Create < Reform::Form
    property :title
    property :abstract
    property :details
    property :pitch
    property :session_format_id
    property :event_id
    property :current_user, virtual: true
    validates :current_user, presence: true
    validates :title, presence: true
    validates :abstract, presence: true
    validates :session_format_id, presence: true
    [...]

We can add parsing default validation errors like this:

# app/controllers/proposals_controller.rb
def create
  result = Proposal::Operation::Create.call(params: params, current_user: current_user)
  if result.success?
    flash[:info] = setup_flash_message(result[:model].event)
    redirect_to event_proposal_url(event_slug: result[:model].event.slug, uuid: result[:model])
  elsif result[:errors] == 'Event not found'
    flash[:danger] = 'Your event could not be found, please check the url.'
    redirect_to events_path
  elsif result[:errors] == 'Event is closed'
    flash[:danger] = 'The CFP is closed for proposal submissions.'
    redirect_to event_path(slug: result[:model].event.slug)
  elsif result["contract.default"].errors.size.positive?
    errors = result["contract.default"].errors.messages
    flash[:danger] = errors.map{ |field, msg| "#{field.to_s.capitalize} => #{msg.first}" }.join(', ')
    redirect_to event_path(slug: result[:model].event.slug)
  end
end

And it will now pass:


bundle exec rspec spec/features/proposal_spec.rb:52
Run options: include {:locations=>{"./spec/features/proposal_spec.rb"=>[52]}}
.

Finished in 2.05 seconds (files took 3.28 seconds to load)
1 example, 0 failures

BUT! We added operations to not deal with those stuff in the controller, so let us parse those errors in operation, and use the same way of displaying errors in the operation as we used to.

One more try. Let's move back to operation


# app/concepts/proposal/operation/create.rb
[...]
step Contract::Validate(key: :proposal)
fail :validation_failed, fail_fast: true

[...]
def validation_failed(ctx, **)
  errors = ctx["contract.default"].errors.messages
  ctx[:errors] = errors.map{ |field, msg| "#{field.to_s.capitalize} => #{msg.first}" }.join(', ')
end
[...]

And then in the controller:

[...]
else
  flash[:danger] = result[:errors]
  redirect_to event_path(slug: result[:model].event.slug)
end
[...]

No nasty mapping in the controller, operation deals with everything, controllers stay a happy place. The test stays green. We change other tests that validate some fields the same way, just to reflect new message display style, validations stay the same, they just happen on contract, not model level.

A plugin that allows rspec for cells is available separately, but if you are following this tutorial you will have it included as a dependency already ¹. We will also rely on capybara since it is the way it has been established in this app so far for making integration/feature tests.

So if you use rspec, be sure to add the gem for it to work with cells
gem "rspec-cells"²

└──● rails g rspec:cell proposal index

Running this will create:

# spec/cells/proposal_cell_spec.rb
require 'rails_helper'

RSpec.describe Proposal::Cell::Index, type: :cell do
  context 'cell rendering' do
    context 'rendering index' do
      subject { cell(described_class, proposal).call(:show) }

      it { is_expected.to have_selector('h1', text: 'Proposal#index') }
      it { is_expected.to have_selector('p', text: 'Find me in app/concepts/proposal/cell/index.haml') }
    end
  end
end

This is quite convenient, although it generates the path as not for the full trailblazer stack, but rather for just using cells as a view object template. We just have to change the class name and set the right subject. Let's add some stuff to the test. Please note that we have access to both cell and concept helpers from the test level. We can recreate the real environment easily thanks to that. Using options we discussed earlier is also available thanks to that (like collection).

We can start taking controller tests and feature tests that touched the index page of proposals and start making them into feature/integration tests from the level of our cell test. In the end, we will be able to get rid of the controller test and one huge feature test in favor of grouped by cell tests. We are also gonna add cells to create action in our controller to complete refactoring that flow to TB. This will also allow us to move some scenarios from the test above.

# app/controllers/proposals_controller.rb
if @event.closed?
  redirect_to event_path(@event)
  flash[:danger] = 'The CFP is closed for proposal submissions.'
  return
end

render html: cell(Proposal::Cell::New, @event,
  context: { current_user: current_user },
  layout: Cfp::Cell::BaseLayout
)
flash.now[:warning] = incomplete_profile_msg unless current_user.complete?
end

if @event.closed?
  redirect_to event_path(@event)
  flash[:danger] = 'The CFP is closed for proposal submissions.'
  return
end
@proposal = @event.proposals.new
@proposal.speakers.build(user: current_user)
flash.now[:warning] = incomplete_profile_msg unless current_user.complete?

We move to set the proposal and building speakers into a cell, but this whole method should just render a form, right? And as you can see in the old controller code, the proposal is being initialized, along with speakers before the render. We also want to initialize and build those stuff, but we don't want to do it in the cell. Of course, we could, but it wouldn't be ideal. Look at the code below, where everything is working and overall seems okay, bu we simply move initialization of the proposal into a cell.

app/concepts/proposal/view/new.haml
.event-info-bar
  .row
    .col-md-8
      .event-info.event-info-dense
        %strong.event-title= event.name
        - if has_date_range?
          %span.event-meta
            %i.fa.fa-fw.fa-calendar
            = date_range
    .col-md-4.text-right.text-right-responsive
      .event-info.event-info-dense
        %span{:class => "event-meta event-status-badge event-status-#{status}"}
          CFP
          = status
        - if event.open?
          %span.event-meta
            CFP closes:
            %strong= closes_at(:month_day_year)

.page-header.page-header-slim
  .row
    .col-md-12
      %h1 Submit a Proposal

.row
  .col-md-12
    .tabbable
      %ul.nav.nav-tabs
        %li.active
          %a{"data-toggle" => "tab", :href => "#create-proposal"} Proposal Form
        %li
          %a{"data-toggle" => "tab", :href => "#preview"} Preview
      .tab-content
        #create-proposal.tab-pane.active
          .row
            .col-md-8
              %p
                Read the <strong>#{link_to 'guidelines', event_path(event.slug)}</strong> to maximize
                your chance of approval. Refrain from including any information that
                would allow a reviewer to identify you.
              %p
                All fields support
                %a{href: markdown_link}
                  %strong GitHub Flavored Markdown.

              = simple_form_for proposal, url: event_proposals_path(event) do |f|
                = render partial: 'form', locals: {f: f}
        #preview.tab-pane
          = render partial: 'preview', locals: { proposal: proposal }

# app/concepts/proposal/cell/new.rb

module Proposal::Cell
  class New < Trailblazer::Cell
    alias event model

    property :start_date
    property :end_date
    property :status
    property :closes_at
    property :date_range

    private

    def has_date_range?
      date_range && end_date
    end

    def markdown_link
      'https://help.github.com/articles/github-flavored-markdown'
    end

    def proposal
      proposal = event.proposals.new
      # This sort of thing should happen in Present scope of operation, we will get to that soon
      proposal.speakers.build(user: context[:current_user])
      proposal
    end
  end
end

We move to set the proposal and building speakers into a cell, but this whole method should just render a form, right? And as you can see in the old controller code, the proposal is being initialized, along with speakers before the render. We also want to initialize and build those stuff, but we don't want to do it in the cell. Of course, we could, but it wouldn't be ideal. Look at the code below, where everything is working and overall seems okay, but we simply move the initialization of the proposal into a cell.

# app/concepts/proposal/operation/create.rb
module Proposal::Operation
  class Create < Trailblazer::Operation
    class Present < Trailblazer::Operation
      step :event
      fail :event_not_found, fail_fast: true
      step Model(Proposal, :new)
      step :event_open?
      fail :event_not_open_error, fail_fast: true
      step :assign_event
      step Contract::Build(constant: Proposal::Contract::Create, builder: -> (ctx, constant:, model:,  **){
        constant.new(model, current_user: ctx[:current_user])
        })
      # [...we will need to move the methods used here...]
    end
    step Contract::Validate(key: :proposal)
    fail :validation_failed, fail_fast: true
    step Contract::Persist(method: :save)
    step :update_user_bio
    [...]

Structure of our operations changes a bit, we encapsulate staff needed before actual creation to prepare a form with a view in the Present class that we then use in new, like this:


.event-info-bar
  .row
    .col-md-8
      .event-info.event-info-dense
        %strong.event-title= event.name
        - if has_date_range?
          %span.event-meta
            %i.fa.fa-fw.fa-calendar
            = event_date_range
    .col-md-4.text-right.text-right-responsive
      .event-info.event-info-dense
        %span{:class => "event-meta event-status-badge event-status-#{event_status}"}
          CFP
          = event_status
        - if event.open?
          %span.event-meta
            CFP closes:
            %strong= event_closes_at

.page-header.page-header-slim
  .row
    .col-md-12
      %h1 Submit a Proposal

.row
  .col-md-12
    .tabbable
      %ul.nav.nav-tabs
        %li.active
          %a{"data-toggle" => "tab", :href => "#create-proposal"} Proposal Form
        %li
          %a{"data-toggle" => "tab", :href => "#preview"} Preview
      .tab-content
        #create-proposal.tab-pane.active
          .row
            .col-md-8
              %p
                Read the <strong>#{link_to 'guidelines', event_path(event.slug)}</strong> to maximize
                your chance of approval. Refrain from including any information that
                would allow a reviewer to identify you.
              %p
                All fields support
                %a{href: markdown_link}
                  %strong GitHub Flavored Markdown.

                =cell(Proposal::Cell::Form, proposal.model, event: event)
        #preview.tab-pane
          =cell(Proposal::Cell::Preview, proposal, event: event)


module Proposal::Cell
  class New < Trailblazer::Cell
    alias proposal model

    private

    def has_date_range?
      event_start_date && event_end_date
    end

    def markdown_link
      'https://help.github.com/articles/github-flavored-markdown'
    end

    def event
      options[:event]
    end

    def event_start_date
      event.start_date
    end
    def event_end_date
      event.end_date
    end

    def event_status
      event.status
    end


    def event_closes_at
      event.closes_at(:month_day_year)
    end

    def event_date_range
      event.date_range
    end
  end
end

Both work, but the second simply makes more sense. Remember not to settle for the first thing that works, but for what works the best and is the most logical or easy to understand. Now that we ended up with this view and this cell, we have to go deeper and remake the partials that were in the new.htaml.haml into cells, creating out the first form in cells. To make the form work in cells, you need to add some include's' to your form cell.

include ActionView::RecordIdentifier
include ActionView::Helpers::FormOptionsHelper
include SimpleForm::ActionViewExtensions::FormHelper

They allow you to use rails form and SimpleForm helpers so be sure to add this. The view will break if you don't add those and use form helpers so it shouldn't be a problem. In the form cell, we can use our form from the view, no problem. We can define form fields in the cells, we can pass the form as an argument to a cell method. As in the examples below.

We can, however, go for more universal form, rather than just doing both new and update with basically the same views.


# app/concepts/proposal/cell/form.rb
module Proposal::Cell
  class Form < Trailblazer::Cell
    include ActionView::RecordIdentifier
    include ActionView::Helpers::FormOptionsHelper
    include SimpleForm::ActionViewExtensions::FormHelper

    property :session_format_name
    property :tags

    def show
      render :form
    end

    private

    def title_input(form)
      form.input :title,
                 autofocus: true,
                 maxlength: :lookup, input_html: { class: 'watched js-maxlength-alert' },
                 hint: 'Publicly viewable title. Ideally catchy, interesting, essence of the talk. Limited to 60 characters.'
    end

    def markdown_link
      'https://help.github.com/articles/github-flavored-markdown'
    end

    def no_track_name_for_speakers
      "#{Track::NO_TRACK} - No Suggested Track"
    end

    def session_format_more_than_one?
      opts_session_formats.length > 1
    end

    def track_name_for_speakers
      proposal.track.try(:name) || no_track_name_for_speakers
    end

    def has_reviewer_activity?
      proposal.ratings.present? || proposal.has_reviewer_comments?
    end

    def multiple_tracks_without_activity?
      event.multiple_tracks? && !has_reviewer_activity?
    end

    def opts_session_formats
      event.session_formats.publicly_viewable.map { |st| [st.name, st.id] }
    end

    def opts_tracks
      event.tracks.sort_by_name.map { |t| [t.name, t.id] }
    end

    def abstract_tooltip
      'A concise, engaging description for the public program. Limited to 600 characters.'
    end

    def abstract_input(form, _abstract_tooltip = 'Proposal Abstract')
      form.input :abstract,
                 maxlength: 1000, input_html: { class: 'watched js-maxlength-alert', rows: 5 },
                 hint: 'A concise, engaging description for the public program. Limited to 1000 characters.' # , popover_icon: { content: tooltip }
    end

    def event
      options[:event]
    end
  end
end


# app/concepts/proposal/cell/new.rb
module Proposal::Cell
  class New < Form
    alias proposal model
    private

    def has_date_range?
      event_start_date && event_end_date
    end

    def event
      options[:event]
    end

    def event_start_date
      event.start_date
    end

    def event_end_date
      event.end_date
    end

    def event_status
      event.status
    end

    def event_closes_at
      event.closes_at.to_s(:month_day_year)
    end

    def event_date_range
      if (event.start_date.month == event.end_date.month) && (event.start_date.day != event.end_date.day)
        event.start_date.strftime('%b %d') + event.end_date.strftime(" \- %d, %Y")
      elsif (event.start_date.month == event.end_date.month) && (event.start_date.day == event.end_date.day)
        event.start_date.strftime('%b %d, %Y')
      else
        event.start_date.strftime('%b %d') + event.end_date.strftime(" \- %b %d, %Y")
      end
    end
  end
end

We just inherit the cell and define view specific methods on the child cell. The same will happen to update, and we are free to use one Form cell as a base for both views. Simple. We do not create a view for both create and update we just do one form.haml view (this, of course, depends on how different your edit and new views would be, here we assume it's the same form, but in the edit, the fields are prefilled).


/* app/concepts/proposal/view/form.haml */
.event-info-bar
  .row
    .col-md-8
      .event-info.event-info-dense
        %strong.event-title= event.name
        - if has_date_range?
          %span.event-meta
            %i.fa.fa-fw.fa-calendar
            = event_date_range
    .col-md-4.text-right.text-right-responsive
      .event-info.event-info-dense
        %span{:class => "event-meta event-status-badge event-status-# {event_status}"}
          CFP
          = event_status
        - if event.open?
          %span.event-meta
            CFP closes:
            %strong= event_closes_at

.page-header.page-header-slim
  .row
    .col-md-12
      %h1 Submit a Proposal

.row
  .col-md-12
    .tabbable
      %ul.nav.nav-tabs
        %li.active
          %a{"data-toggle" => "tab", :href => "#create-proposal"} Proposal Form
        %li
          %a{"data-toggle" => "tab", :href => "#preview"} Preview
      .tab-content
        #create-proposal.tab-pane.active
          .row
            .col-md-8
              %p
                Read the <strong>#{link_to 'guidelines', event_path(event.slug)}</strong> to maximize
                your chance of approval. Refrain from including any information that
                would allow a reviewer to identify you.
              %p
                All fields support
                %a{href: markdown_link}
                  %strong GitHub Flavored Markdown.
                = simple_form_for proposal, url: event_proposals_path(event) do |f|
                  %fieldset.margin-top
                    = title_input(f)

                    - if !has_reviewer_activity?
                      - if session_format_more_than_one?
                        = f.association :session_format, collection: opts_session_formats, include_blank: 'None selected', required: true, input_html: {class: 'dropdown'},
                          hint: "The format your proposal will follow."#, popover_icon: { content: session_format_tooltip }
                      - else
                        = f.association :session_format, collection: opts_session_formats, include_blank: false, input_html: {readonly: "readonly"},
                          hint: "The format your proposal will follow."#, popover_icon: { content: "Only One Session Format for #{event.name}" }
                    - else
                      .form-group
                        = f.label :session_format, 'Session format'
                        %div #{session_format_name}

                    - if multiple_tracks_without_activity?
                      = f.association :track, collection: opts_tracks, include_blank: no_track_name_for_speakers, input_html: {class: 'dropdown'},
                        hint: "Optional: suggest a specific track to be considered for."#, popover_icon: { content: track_tooltip }
                    - else
                      .form-group
                        = f.label :track, 'Track'
                        %div #{track_name_for_speakers}

                    = abstract_input(f, abstract_tooltip)

                    - if event.public_tags?
                      .form-group
                        %h3.control-label
                          Tags
                        = f.select :tags,
                          options_for_select(event.proposal_tags, tags),
                          {}, {class: 'multiselect proposal-tags', multiple: true }

                  %fieldset
                    %legend.fieldset-legend For Review Committee
                    %p
                      This content will <strong> only</strong> be visible to the review committee.

                    = f.input :details, input_html: { class: 'watched', rows: 5 },
                      hint: 'Include any pertinent details such as outlines, outcomes or intended audience.'#, popover_icon: { content: details_tooltip }

                    = f.input :pitch, input_html: { class: 'watched', rows: 5 },
                      hint: 'Explain why this talk should be considered and what makes you qualified to speak on the topic.'#, popover_icon: { content: pitch_tooltip }

                  - if event.custom_fields.any?
                    - event.custom_fields.each do |custom_field|1
                      .form-group
                        = f.label custom_field
                        = text_field_tag "proposal[custom_fields][#{custom_field}]", proposal.custom_fields[custom_field], class: "form-control"

                  -# TODO
                  -# = render partial: 'speakers/fields', locals: { f: f, event: event }

                  .form-submit.clearfix.text-right
                    - if proposal.persisted?
                      = link_to "Cancel", event_proposal_path(event_slug: event.slug, uuid: proposal), {class: "btn btn-default btn-lg"}
                    - else
                      = link_to "Cancel", event_path(event.slug), {class: "btn btn-default btn-lg"}
                    %button.btn.btn-primary.btn-lg{type: "submit"} Submit
        #preview.tab-pane
          =cell(Proposal::Cell::Preview, proposal, event: event)

This will render our form view:

We defined some fields and cells, and put a lot of helper methods in it, we could move more and more to cells, but I think you get the gist by now of what you can do with it.

Let's finish our cell test.

require 'rails_helper'

RSpec.describe Proposal::Cell::Index, type: :cell do
  subject { cell(described_class, current_user, context: { current_user: current_user }).call(:show) }
  controller ProposalsController
  context 'cell rendering' do
    context 'without proposals' do
      let(:current_user) { FactoryGirl.create(:user) }
      it { expect(subject).to have_content("You don't have any proposals.") }
    end

    context 'with proposals' do
      let(:current_user) { FactoryGirl.create(:user) }
      let!(:proposal) { create :proposal }
      let!(:speaker) { create :speaker, user: current_user, proposal: proposal, event: proposal.event }

      it { expect(subject).to have_content proposal.title }
    end
  end
end

This is our test in the base, simplistic form (just to show the syntax basically, test content with expects, etc. is not relevant here). Please note, that we do have a controller dependency in this cell test, without controller ProposalsController we would get an error in this particular case (the first test, passes without the dependency, second does not cause one of the nested cells needs the controller)³.

Join us next time, when we cover more and more aspects of Trailblazer!

GH: https://github.com/trailblazer/rspec-cells documentation from TB: http://trailblazer.to/gems/cells/testing.html ↩
Make sure to add this gem in the gemfile AFTER capybara (below it, in order). https://github.com/trailblazer/rspec-cells/issues/21 ↩
https://github.com/trailblazer/cells/wiki/Troubleshooting-Tests ↩

Trailblazer tutorial: refactoring legacy rails views with Trailblazer Cells - part 5

Krzysztof — Thu, 07 Nov 2019 11:44:11 +0000

Encapsulating backend business logic into Trailblazer operations is a very important part of building an app based on it, alongside contracts and moving logic from controllers. Another important step that ties everything from backend to frontend together, are cells¹. Let's quickly go over what cells are, why we should use them and then we will find some view that could use refactoring and help him be a better, clearer view.

As explained by the cell gem webpage

[...] cells, are simple ruby classes that can render templates.

Sounds nice and simple, right? Well... that's actually cause it's nice and simple. Their base functionality is nothing complicated and that's what we will cover in this post, but going further we will keep in mind cells can be used in so many more ways, than a simple template renderer². Let's pick a view, and to be consistent with this series, we will simply go to Proposals index view. Here's how it looks on the webpage:

And here is the code we are dealing with:

/* app/views/proposals/index.html.haml */
.row
  .col-md-12
    .page-header
      %h1 My Proposals

.row
  .col-md-12.proposals
    - if proposals.blank? && invitations.blank?
      .widget.widget-card.text-center
        %h2.control-label You don't have any proposals.

    - events.each do |event|
      - talks = proposals[event] || []
      - invites = invitations[event] || []
      - event = event.decorate
      .row
        .col-md-4
          .widget.widget-card.flush-top
            .event-info.event-info-block
              %strong.event-title
                = event.name
              %span{:class => "event-status-badge event-status-#{event.status}"}
                CFP
                = event.status
              %span.pull-right
                - if event.open?
                  = link_to 'Submit a proposal', new_event_proposal_path(event.slug), class: 'btn btn-primary btn-sm'
              .event-meta
                - if event.start_date? && event.end_date?
                  %span.event-meta-item
                    %i.fa.fa-fw.fa-calendar
                    = event.date_range
              .event-meta.margin-top
                %span.event-meta
                  = link_to 'View Guidelines', event_path(event.slug)

        .col-md-8
          - if invites.present?
            .proposal-section.invitations.callout
              %h2.callout-title Speaker Invitations
              %ul.list-unstyled
                - invites.each do |invitation|
                  %li.invitation.proposal.proposal-info-bar
                    .flex-container.flex-container-md
                      .flex-item.flex-item-padded
                        %h4.proposal-title= link_to invitation.proposal.title, invitation_path(invitation.slug)

                        .proposal-meta.proposal-description
                          .proposal-meta-item
                            %strong Track:
                            = invitation.proposal.track_name
                          .proposal-meta-item
                            %strong #{ 'Speaker'.pluralize(invitation.proposal.speakers.count) }:
                            = invitation.proposal.speakers.collect { |speaker| speaker.name }.join(', ')
                      .flex-item.flex-item-fixed.flex-item-padded.flex-item-right
                        .invitation-status
                          = invitation.state_label
                        - if invitation.pending?
                          .proposal-meta.invite-btns
                            = invitation.decline_button(small: true)

                            = invitation.accept_button(small: true)

          .proposal-section
            %ul.list-unstyled
              - talks.each do |proposal|
                %li.proposal.proposal-info-bar
                  .flex-container.flex-container-md
                    .flex-item.flex-item-fixed.flex-item-padded.proposal-icon
                      %i.fa.fa-fw.fa-file-text
                    .flex-item.flex-item-padded
                      %h4.proposal-title= link_to proposal.title, event_proposal_path(event_slug: proposal.event.slug, uuid: proposal)
                      .proposal-meta.proposal-description
                        .proposal-meta-item
                          %strong #{ 'Speaker'.pluralize(proposal.speakers.count) }:
                          %span= proposal.speakers.collect { |speaker| speaker.name }.join(', ')
                        .proposal-meta-item
                          %strong Format:
                          %span #{proposal.session_format_name}

                        .proposal-meta-item
                          %strong Track:
                          %span #{proposal.track_name}
                      .proposal-meta.margin-top
                        %strong Updated:
                        %span #{proposal.updated_in_words}
                    .flex-item.flex-item-fixed.flex-item-padded
                      .proposal-status
                        = proposal.speaker_state(small: true)
                      .proposal-meta
                        %i.fa.fa-fw.fa-comments
                        = pluralize(proposal.public_comments.count, 'comment')

A rather big view, isn't? Good thing it is using a decorator (with Draper gem in this case) which is a good practice and will make it easier for us to improve it even more than simply adding a decorator. First off we need to add some base cell and render it from the controller level. In our concepts we already have a concept for proposal, we just need to add two folders there: cells and views. This is an important concept in TB, to keep everything connected to our concept in its folder. Thanks to that we won't have a humongous views folder where we have to look for adequate view, we will know that if we want a view for proposals, we simply go into its concept, and into view folder there, where we will have all the relative views bundled up.

Rendering cell is rather simple but gives a lot of options. As per version 2.1 the syntax for invoking a cell is either this

=concept("your_concept/cell", your_model)

(which is a syntax for using cells as a solo gem, as you will see further down below, using it with full trailblazer offers, different, more complex helper with more options), or:

  YourConcept::Cell.new(your_model).show

Please note that the haml example is called from the view, and what it does is invoking the second, ruby syntax, it's just a helper. The helper will always invoke the show method which is a default method of returning the cell body. Also, keep in mind that I called the passed object your_model for two reasons.

Whatever you pass into the helper in the place of your_model will be called model inside the cell body
Your passed your_model does not have to be an AR object, you can use cell on other types of data

After adding gem 'trailblazer-cells' we also need to add gem 'cells-hamlit' gem 'cells-rails' to have cells working with Trailblazer, instead of as a solo abstraction for view rendering, where we need only "cells" gem. Now we can start by adding a new class, a new base view, and render it from the controller level. Then step by step we will move stuff from the current view to the new one, using cells.

# app/controllers/proposals_controller.rb
def index
  proposals = current_user.proposals.decorate.group_by {|p| p.event}
  invitations = current_user.pending_invitations.decorate.group_by {|inv| inv.proposal.event}
  events = (proposals.keys | invitations.keys).uniq

  render locals: {
      events: events,
      proposals: proposals,
      invitations: invitations
  }
end

This is a base class that we will render in the controller.

# app/concepts/proposal/cell/index.rb
module Proposal::Cell
  class Index < Trailblazer::Cell
  end
end

We will only invoke this in our index action:

# app/controllers/proposals_controller.rb
def index
  render html: cell(Proposal::Cell::Index, current_user, context: { current_user: current_user })
end

Using the cell helper we pass in a cell we want to render, our model (in this case its user), and options (in this case nothing yet)
We are passing in the current user as our model, cause it is our model in this case, but we also pass it in context. This will get useful later on cause what we pass in in context, also gets passed to our cell nested cells. By default, we have controller data passed in context, so that we can have access to stuff like our current path, or CRUD action name (this will be used later on to demonstrate where and how we can use context).

This alone with a base view made in app/concepts/proposal/view/index.haml like this:

# app/concepts/proposal/view/index.haml
.row
  .col-md-12
    .page-header
      %h1 My Proposals

...will give us a result


  "<div class='row'>\n<div class='col-md-12'>\n<div class='page-header'>\n<h1>My Proposals</h1>\n</div>\n</div>\n</div>\n"

Nothing pretty but gives us ground to work on. Developing cell further on we add what we previously had passed in as locals for the view.

# app/concepts/proposal/cell/index.rb
module Proposal::Cell
  class Index < Trailblazer::Cell
    alias user model

    private

    def proposals
      user.proposals.decorate.group_by(&:event)
    end

    def no_proposals?
      proposals.blank? && invitations.blank?
    end

    def invitations
      user.pending_invitations.decorate.group_by { |inv| inv.proposal.event }
    end

    def events
      (proposals.keys | invitations.keys).uniq
    end
  end
end

With this, we already have access to what we had while dealing with this logic in the controller. We can now move on with recreating the view TB way.

/* app/concepts/proposal/view/index.haml */
.row
  .col-md-12
    .page-header
      %h1 My Proposals
.row
  .col-md-12.proposals
    - if no_proposals?
      .widget.widget-card.text-center
        %h2.control-label You don't have any proposals.

    - events.each do |event|
      = cell(Proposal::Cell::EventRow, event, current_user: user, proposals: proposals, invitations: invitations)

We start off standard, we move the condition to a separate method, that we name according to what the condition checks. Then we go over proposal events, and iterate over them, rendering a cell for each of those events. This is a pattern you will see more in our guides. Instead of using partials, helpers, etc. we only used nested cells, that we render for each object, keeping their logic and code nicely separated. We also pass some additional stuff in as options to the event cell.

The controller should not deal with finding those objects if possible, this is something cells can do for us. As we mentioned before, a decorator gem is used in the system to help with the views logic, we will try to move away from it and transfer everything into correct cells. So we create some more "helper" methods and we end up with this new cell for each event we display in our proposals view.

# app/concepts/proposal/cell/event_row.rb
module Proposal::Cell
  class EventRow < Trailblazer::Cell
    alias event model

    private

    def talks
      options[:proposals][event] || []
    end

    def invites
      options[:invitations][event] || []
    end

    def has_date_range?
      event.start_date? && event.end_date?
    end

    def status_class
      "event-status-badge event-status-#{event.status}"
    end

    def new_event_link
      link_to(
        new_event_proposal_path(event.slug),
        'Submit a proposal',
        class: 'btn btn-primary btn-sm'
      )
    end

    def guidelines_link
      link_to(event_path(event.slug), 'View Guidelines')
    end

    def date_range
      if (event.start_date.month == event.end_date.month) && (event.start_date.day != event.end_date.day)
        event.start_date.strftime('%b %d') + event.end_date.strftime(" \- %d, %Y")
      elsif (event.start_date.month == event.end_date.month) && (event.start_date.day == event.end_date.day)
        event.start_date.strftime('%b %d, %Y')
      else
        event.start_date.strftime('%b %d') + object.end_date.strftime(" \- %b %d, %Y")
      end
    end
  end
end

This gives us this view:

# app/concepts/proposal/view/event_row.haml
.row
  .col-md-4
    .widget.widget-card.flush-top
      .event-info.event-info-block
        %strong.event-title
          = event.name
        %span{:class => status_class}
          CFP
          = event.status
        %span.pull-right
          - if event.open?
            = new_event_link
        .event-meta
          -if has_date_range?
            %span.event-meta-item
              %i.fa.fa-fw.fa-calendar
              = date_range
        .event-meta.margin-top
          %span.event-meta
            = guidelines_link

We moved logic and everything connected to it to the cell. When you take a look at the original view, you will see that we have more nested views. This is perfect for cells (and should have probably been done with partials in the first place, but luckily we are here to do it the right way). Let's make adequate cells for invites and talks (they were previously set in the view, this also has been changed to use cells to assign those in the classes rather than the view) and render them in each event cell iteration.

In invite row I would like to point out another useful thing:

/* app/concepts/proposal/view/invite_row.haml */
%li.invitation.proposal.proposal-info-bar
  .flex-container.flex-container-md
    .flex-item.flex-item-padded
      %h4.proposal-title
        = invitation_link

      .proposal-meta.proposal-description
        .proposal-meta-item
          %strong Track:
          = track_name
        .proposal-meta-item
          %strong
            = title
          = speakers
    .flex-item.flex-item-fixed.flex-item-padded.flex-item-right
      .invitation-status
        =  content_tag :span, state, class: label_class
      - if invite.pending?
        .proposal-meta.invite-btns
          = decline_button(small: true)

          = accept_button(small: true)

/* app/concepts/proposal/cell/invite_row.rb */

module Proposal::Cell
  class InviteRow < Trailblazer::Cell
    alias invite model
    property :state
    property :proposal
    property :slug

    STATE_LABEL_MAP = {
      Invitation::State::PENDING => 'label-default',
      Invitation::State::DECLINED => 'label-danger',
      Invitation::State::ACCEPTED => 'label-success'
    }.freeze

    private

    def invitation_link
      link_to(invite.proposal.title, invitation_path(invite.slug))
    end

    def speakers
      invite.proposal.speakers.collect(&:name).join(', ')
    end

    def title
      'Speaker'.pluralize(invite.proposal.speakers.count)
    end

    def track_name
      proposal.track.try(:name) || Track::NO_TRACK
    end

    def label_class
      "label #{STATE_LABEL_MAP[state]}"
    end

    def decline_button(small: false)
      classes = 'btn btn-danger'
      classes += ' btn-xs' if small

      link_to 'Decline', decline_invitation_path(slug), class: classes,
                                                        data: { confirm: 'Are you sure you want to decline this invitation?' }
    end

    def accept_button(small: false)
      classes = 'btn btn-success'
      classes += ' btn-xs' if small

      link_to 'Accept', accept_invitation_path(slug), class: classes
    end
  end
end

property allows us to more easily access our model fields, while also nicely specifying them in the cell. So in the view = invitation.slug becomes simply slug cause we already know we are in the invitation "scope" and that scope has a proposal property. Going back we can also use this in EventRow, see for yourself if you can spot the lines where we can delete the event.something and replace it by just something by adding adequate property.

As you can see on the decline_button and accept_button methods we can easily add html elements in the cell. Also knowing the rules by now, and having some examples and seeing the code, try building the TalkRow cell and view on your own. My version is just below, but it might be a good check of your so far understanding of the topic.

 # app/concepts/proposal/cell/talk_row.rb
 module Proposal::Cell
  class TalkRow < Trailblazer::Cell
    alias talk model
    property :title
    property :event
    property :speakers
    property :session_format_name
    property :track_name
    property :updated_in_words
    property :public_comments

    private

    def talk_link
      link_to(title, event_proposal_path(event_slug: options[:event].slug, uuid: talk))
    end

    def title
      'Speaker'.pluralize(speakers.size)
    end

    def speakers_collection
      speakers.collect(&:name).join(', ')
    end

    def comments
      pluralize(public_comments.size, 'comment')
    end
  end
 end

/* app/concepts/proposal/view/talk_row.haml */
%li.proposal.proposal-info-bar
  .flex-container.flex-container-md
    .flex-item.flex-item-fixed.flex-item-padded.proposal-icon
      %i.fa.fa-fw.fa-file-text
    .flex-item.flex-item-padded
      %h4.proposal-title
        = talk_link
      .proposal-meta.proposal-description
        .proposal-meta-item
          %strong
            = title
          %span
            = speakers_collection
        .proposal-meta-item
          %strong Format:
          %span
            = session_format_name
        .proposal-meta-item
          %strong Track:
          %span
            = track_name
      .proposal-meta.margin-top
        %strong Updated:
        %span
          = updated_in_words
    .flex-item.flex-item-fixed.flex-item-padded
      .proposal-status
        = talk.speaker_state(small: true)
      .proposal-meta
        %i.fa.fa-fw.fa-comments
        = comments

Okay, seems like we are done? We can delete the index.html.haml file and check out how our page looks again!

What are we forgetting? Why is it so basic? Well, when using standard rails views we have an app/views/layouts/application.html.haml to give us a base to build around, with not only base html elements, but also including styles, javascript, nav bar, footer, etc... we also need a cell for that. So let's build it quickly. Even though this still be a cell we should call it a Layout.

# app/concepts/cfp/cell/base_layout.rb
module Cfp
  module Cell
    class BaseLayout < Trailblazer::Cell
    end
  end
end

Cfp, by the way, is the name of our app. Base layout view can look something like this:

/* app/concepts/cfp/view/base_layout.haml */
!!!5
%html(lang="en")
  %head
    %title= "Cfp"
    %meta(name="viewport" content="width=device-width, initial-scale=1.0")

    = stylesheet_link_tag 'application', media: 'all'
    = stylesheet_link_tag '//maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css', media: 'all'
    = stylesheet_link_tag "//fonts.googleapis.com/css?family=Open+Sans:400italic,600italic,400,600"

    = javascript_include_tag 'application'
    = javascript_include_tag "//www.google.com/jsapi", "chartkick"
    = javascript_include_tag { yield :custom_js }

    // html5 shim and respond.js ie8 support of html5 elements and media queries
    /[if lt ie 9]
      = javascript_include_tag "/js/html5shiv.js"
      = javascript_include_tag "/js/respond.min.js"

  -# %body{id: body_id}
  -#   = render partial: "layouts/navbar"

  -# #flash= show_flash
  %body
    .main
      .main-inner
        .container-fluid
          =yield

  -# = render partial: "layouts/event_footer"

As you can see we have some comments here. Well get to them but first, let us just talk about why the layout is important. We should edit our controller action to use the base layout:

# app/controllers/proposals_controller.rb
def index
  render html: cell(Proposal::Cell::Index, current_user, context: { current_user: current_user }, layout: Cfp::Cell::BaseLayout)
end

And we get the content of our index page back.

Without nav bar though. This is a job for another cell since the nav bar is something that will get repeated every time we render the base layout. So we simply create a NavBar cell in the cfp concept, and add the view there, we render it each time with our BaseLayout and we yield the content of the cell that we are using the layout for (so Proposal::Cell::Index in our case). While we are at it, we might as well create the same thing for event_footer partial (the last comment in the haml code above). The problem with nav_bar and event_footer is that they rely heavily on ApplicationHelper and ApplicationController. Ideally, we want to skim those of methods that can be dealt with by cells, but we cant delete those from there until we refactor the whole app. For now, we are only doing it for one place, so we will have to define them anew in cells. Let's take a look at nav_bar and its many calls to helper methods, that we will move to cells.

/* app/views/layouts/_navbar.html.haml */
.navbar.navbar-default.navbar-fixed-top
  .container-fluid
    .navbar-header
      %button.navbar-toggle{ type: "button", data: { toggle: "collapse", target: ".navbar-collapse" } }
        %span.icon-bar
        %span.icon-bar
        %span.icon-bar
      - if current_event
        = link_to "#{current_event.name} CFP", event_path(current_event), class: 'navbar-brand'
      - else
        = link_to "CFP App", events_path, class: 'navbar-brand'

    .collapse.navbar-collapse
      - if current_user
        %ul.nav.navbar-nav.navbar-right
          - if speaker_nav?
            %li{class: nav_item_class("my-proposals-link")}
              = link_to proposals_path do
                %i.fa.fa-file-text
                %span My Proposals

          - if review_nav?
            %li{class: nav_item_class("event-review-proposals-link")}
              = link_to event_staff_proposals_path(current_event) do
                %i.fa.fa-balance-scale
                %span Review

          - if program_nav?
            %li{class: nav_item_class("event-program-link")}
              = link_to event_staff_program_proposals_path(current_event) do
                %i.fa.fa-sitemap
                %span Program

          - if schedule_nav?
            %li{class: nav_item_class("event-schedule-link")}
              = link_to event_staff_schedule_grid_path(current_event) do
                %i.fa.fa-calendar
                %span Schedule

          - if staff_nav?
            %li{class: nav_item_class("event-dashboard-link")}
              = link_to event_staff_path(current_event) do
                %i.fa.fa-dashboard
                %span Dashboard

          - if admin_nav?
            = render partial: "layouts/nav/admin_nav"

          = render partial: "layouts/nav/notifications_list"

          = render partial: "layouts/nav/user_dropdown"

      - else
        %ul.nav.navbar-nav.navbar-right
          %li= link_to "Log in", new_user_session_path

- if display_staff_event_subnav?
  = render partial: "layouts/nav/staff/event_subnav"

- elsif program_mode?
  = render partial: "layouts/nav/staff/program_subnav"

- elsif schedule_mode?
  = render partial: "layouts/nav/staff/schedule_subnav"

We can start moving what we can to appropriate places. By now you might be able to visualize the code we will end up with or even DIY if you so wish. If you try you will see that refactoring some legacy rails apps to TB is not that hard, and the flow of it is very fluent and logical. We will also see how we can get rid of some rails callbacks using logic in cells. Navbar in our legacy project is big enough to deserve its concept too, so we are getting more and more of code division and grouping. I will post the result of our base layout and navbar view. We will go over the most interesting and noteworthy stuff and add some to do's for later.

First of all, this is our current stack of files.

As you can see we hold only the base stuff in Cfp namespace, when we use layout: Cfp::Cell::BaseLayout this layout in our controller we get all the stuff that this layout gives us, and when we pass the context: { current_user: current_user } we will get access to context[:current_user] in EVERY cell that follows rendering of base layout. We won't have to pass it every time. So, for example, this is our BaseLayout view and cell right now:

# app/concepts/cfp/cell/base_layout.rb
module Cfp
  module Cell
    class BaseLayout < Trailblazer::Cell
      private

      def current_user
        context[:current_user]
      end

      def body_id
        "#{context[:controller].request.env['REQUEST_PATH'].tr('/', '_')}_#{context[:controller].action_name}"
      end

      def show_flash
        context[:controller].flash.map do |key, value|
          key += ' alert-info' if key == 'notice'
          key = 'danger' if key == 'alert'
          content_tag(:div, class: "container alert alert-dismissable alert-#{key}") do
            content_tag(:button, content_tag(:span, '', class: 'glyphicon glyphicon-remove'),
                        class: 'close', data: { dismiss: 'alert' }) +
              simple_format(value)
          end
        end.join.html_safe
      end

      def current_event
        @current_event ||= set_current_event(session[:current_event_id]) if session[:current_event_id]
      end

      def set_current_event(event_id)
        @current_event = Event.find_by(id: event_id).try(:decorate)
        session[:current_event_id] = @current_event.try(:id)
        @current_event
      end
    end
  end
end

It has some helper methods, it sets current event to be passed further down to EventFooter cell, it uses context[:controller] - set by default by Trailblazer - to deal with flash messages display and classes, and it uses context[:current_user] to set an easier access to current_user via a method rather than accessing context each time. It also uses the controller from context to set some css id.

/* app/concepts/cfp/view/base_layout.haml */
!!!5
%html(lang="en")
  %head
    %title= title
    %meta(name="viewport" content="width=device-width, initial-scale=1.0")

    = stylesheet_link_tag 'application', media: 'all'
    = stylesheet_link_tag '//maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css', media: 'all'
    = stylesheet_link_tag "//fonts.googleapis.com/css?family=Open+Sans:400italic,600italic,400,600"

    = javascript_include_tag 'application'
    = javascript_include_tag "//www.google.com/jsapi", "chartkick"
    = javascript_include_tag { yield :custom_js }

    // html5 shim and respond.js ie8 support of html5 elements and media queries
    /[if lt ie 9]
      = javascript_include_tag "/js/html5shiv.js"
      = javascript_include_tag "/js/respond.min.js"


  #flash= show_flash

  %body{id: body_id}
    = cell(Navigation::Cell::Main, nil, context: {current_event: current_event})
    .main
      .main-inner
        .container-fluid
          =yield

  = cell(Cfp::Cell::EventFooter, nil, context: {current_event: current_event})

In the view, as you can see we do some stuff we usually do in application.html, but doing so in cells gives us more flexibility and makes it easier to create a separate layout for different parts of the system. We will get to that when we create a layout for unlogged users. We also added another Navigation scope in concepts where we deal with nav bar for our view.

/* app/concepts/navigation/view/main.haml */
.navbar.navbar-default.navbar-fixed-top
  .container-fluid
    .navbar-header
      %button.navbar-toggle{ type: "button", data: { toggle: "collapse", target: ".navbar-collapse" } }
        %span.icon-bar
        %span.icon-bar
        %span.icon-bar
      - if current_event
        = link_to "#{current_event.name} CFP", event_path(current_event), class: 'navbar-brand'
      - else
        = link_to "CFP App", events_path, class: 'navbar-brand'

    .collapse.navbar-collapse
      - if current_user
        %ul.nav.navbar-nav.navbar-right
          - if speaker_nav?
            %li{class: nav_item_class("my-proposals-link")}
              %a{:href => proposals_path}
                %i.fa.fa-file-text
                %span My Proposals
          - if review_nav?
            %li{class: nav_item_class("event-review-proposals-link")}
              %a{:href => event_staff_proposals_path(current_event)}
                %i.fa.fa-balance-scale
                %span Review
          - if program_nav?
            %li{class: nav_item_class("event-program-link")}
              %a{:href => event_staff_program_proposals_path(current_event)}
                %i.fa.fa-sitemap
                %span Program
          - if schedule_nav?
            %li{class: nav_item_class("event-schedule-link")}
              %a{:href => event_staff_schedule_grid_path(current_event)}
                %i.fa.fa-calendar
                %span Schedule
          - if staff_nav?
            %li{class: nav_item_class("event-dashboard-link")}
              %a{:href => event_staff_path(current_event)}
                %i.fa.fa-dashboard
                %span Dashboard

          - if admin_nav?
            = cell(Navigation::Cell::Admin)
          = cell(Navigation::Cell::NotificationsList)
          = cell(Navigation::Cell::UserDropdown)

      - else
        %ul.nav.navbar-nav.navbar-right
          %li= link_to "Log in", new_user_session_path

This is the view of our nav bar, it encapsulates the logic for all the ifs in the cell, and it renders more cells. Like NotificationsList.

# app/concepts/navigation/cell/notifications_list.rb
module Navigation
  module Cell
    class NotificationsList < Trailblazer::Cell
      def notifications_count
        context[:current_user].notifications.unread.length
      end

      def notifications_more_unread_count
        context[:current_user].notifications.more_unread_count
      end

      def notifications_unread
        context[:current_user].notifications.recent_unread
      end

      def unread_notications?
        context[:current_user].notifications.unread.any?
      end

      def more_unread_notifications?
        context[:current_user].notifications.more_unread?
      end
    end
  end
end

See? It still uses the context that was passed way back in the controller, we only passed the current user once into the context, into a completely different cell, but we still have easy access to it. Pasting all the small cells and views would be unnecessary, and at this point, you can probably figure them out by yourself. Now lets checkout out our view.

This pretty much covers the base of refactoring legacy views, there are still more options to cells of course, but this is enough to get you started. Next up we will cover testing cells, passing collections into cells, using a different layout for different views and other cells options that will come along the way.

The cells gem is completely stand-alone and can be used without Trailblazer. ↩
As said in the gem page, cells can be used to build a proper OOP, polymorphic builders, nesting, view inheritance, using Rails helpers, asset packaging to bundle JS, CSS or images, simple distribution via gems or Rails engines, encapsulated testing, etc. ↩

Git gud at git - part 1

Krzysztof — Fri, 31 May 2019 10:51:10 +0000

ANOTHER GIT TUTORIAL? WHY? WHY GOD, WHY?

Well for sure there is plenty of git guides etc out there, but from my own experience, I know that most of them are really, really basic and limit themselves to sort of copy/paste documentation and make it shorter. I really think that understanding how git works and what options it gives you requires more attention. Lots of people will throw git commands at you and say what those commands do when you use them, without explaining what really happens in the files while invoking the command. I will strive to provide a thorough understanding of the basic phrases, commands, and concepts that you’ll use frequently in your work with git (and that you are probably using already). It might prove to be a long read split into several articles and if you manage to go through all of them I will be really grateful for reading what I had to say.

I won’t lie that every concept and a word I will talk about in those guides came straight out of my head, I try to learn git more and more every day so I will often double check something I talk about and see what other people had to say about a given subject. I will provide links to all my sources that I use as a knowledge base and to all articles and posts that gave me a better understanding of concepts I talk about.

And finally, if I get something wrong - please be sure to comment on that. As I mentioned I strive to learn more and more and the process of learning is always ongoing while working on this series I hope to learn a lot myself, and if I manage to learn something from my readers then my life would be complete and I can end it all.

FOR THE ROAD

As much as I will try to explain everything along the way, if git is still something somewhat new to you, please make sure you have some reference to not get lost, like an official glossary https://git-scm.com/docs/gitglossary or even complete documentation https://git-scm.com/docs. I know that sending you straight to documentation might sound a bit like missing the point of a tutorial-like article, but if I make a mistake of not explaining something that I talk about, you can at least have an easy to-go reference to catch up on the given subject.

WHERE’S GIT?

A lot of text editors will by default, hide git files, and files ignored from git from you in their tree view. In general, it’s a good thing, but for educational purposes, I will use atom git tree to showcase some of git files structure.

GIT OBJECTS

Let’s get right into it and see what those files look like.
We should maybe start by checking the refs folder. Also, let’s compare it to our current CLI output and git position.

I’ve customized my CLI prompt to always showcase what git branch I am currently at and I suggest you do the same, it’s rather convenient. Anyway, we can see that we are currently in a branch called feature/branch_with_a_lot_of_stuff_to_do. In the file structure, we see a folder called feature and a file name branch_with_a_lot_of_stuff_to_do. Using “/” as a way to structure branches into features, hotfixes and releases is a popular way of naming and organizing branches, and git recognizes it nicely and creates adequate folders for it. What interests us the most right now though is the branch file itself. Let's open the file in a text editor since it’s just a text file.

And now run git log in the terminal.

The only content of the file is actually a checksum, that correlates with our current branch checksum. Still, it is a little bit disappointing that that's all there is, right? Where is the actual content, not just the pointers?
Let’s find out more about our commit, where did it come from, are where can it lead us.

cat-file commit might be a new command, but it can prove useful it provides us with more detailed info about git commit object. There are some other objects we need to learn about.

Here’s the view of object folder for where we will be looking for some objects.
Okay, so we have 3 git objects to discuss, blobs, trees, and commits. Once we get through that, we'll know the basics of git inner workings. We should start from the top, and at the top we have commit.

Commit as we saw thanks to cat-file commit contains mostly pointers and some message that we put in it. Pointers are once again cheksums, and the one that is the most interesting one for us right now is the tree pointer. So let’s go to our objects folder and look for the pointed tree. To find it we need to take the first two signs of the tree checksum (ca) and look for the folder called the same way. Check out the screenshot with folders from above, and you will be able to see it. Inside it, we have another text file, with the rest of the checksum (it’s only missing the first two letters that are in the name of the folder).

Here we investigate our tree with git ls-tree. The result shows us the list of our changed files, directories, and points us to their blobs. It also points us to other trees if the changes are nested, while blobs indicate a file has been changed in the current tree directory. This structure is often compared to branches on a tree, and I find it very fitting. Each branch (tree object) has something on it (blobs) but can also lead us further to another branch (tree object) which will have its own blobs and tree pointers. I hope this is clear enough.

Finally, we get to see our blobs which contain the actual changes to the files. I’ll just pick one of the blobs listed in the picture above.

Here’s an attempt of visualizing the structure that leads from commit, through the tree, all the way down to changes made in a file.
Let's backtrack a bit and understand the parent pointer that we saw in the commit. What does that mean? If we would want to change one of the blobs that our tree showed us (like the README file) git would create another blob (without overwriting or deleting the old one) and another tree and that tree would have a pointer to the new blob and all the other pointers our first tree has (assuming only one file changed). Then, at last, this tree would need a reference in a commit. This new commit would have a parent commit, which would be our older commit.
And in overall that’s how objects look like, work and correlate with each other in git! If you want more details about those aspects of git, check out the video I link at the end by Scott Chacon co-founder of GitHub.

GIT INDEX, AREAS

The next thing I want to talk about in the first part of those blog posts is “states” or “places” or “areas” in git (I will be using the last description, simply cause I like it the most). This will let us see how commits, blobs, and trees are used and created. Keep in mind though that even we might call them like some dedicated memory assets with directories and files, git doesn't actually have a directory called staging, no working directory (even though we will be using those names and they are accurate to what happens in them). The repository is, on the other hand, our place for the actual code.
The way all of this happens is through a file called index (hence the current header). This file tracks changes made to your files, through all three areas mentioned above. Adding changes to staging area updates the index with changes implemented, creates new blob and puts them in /objects along with other blobs for adequate commit.

Let's go through all of those three areas
Working directory area - this is where you start when you add new lines of code, new files, etc. They remain in this area until you add. Then they got transferred to…

Staging area - On the picture above we see green stuff and red stuff. As you can see above all green files listed we have a prompt saying “Changes to be committed”. That is precisely what the staging area is. It’s all the changes that are gonna be inside your next commit. Red files will be omitted. If you want to learn more about manipulating the data you stage, you can try reading my other blog post about git add --patch.
Committing stage area changes created new files in /objects and we get our commits, blobs, trees. Remember though, that until they are committed, no actual real area is created in git and we are still using the index to store changes and understanding index in detail will be the last thing we do in this blog post but first...

Repository area - directory that holds all our code, that git keeps track of and updates its index based upon changes made to files in this area. In ./git the folders /objects is the place that contains all the pointers to the actual code (in the form of objects we talked about).

So let's talk about how index holds all the information needed.

This is a small part of binary dump for my index file for a small project. This output takes a lot of space and it might be a bit hard to read. Let's try a hex dump version.

Slightly better, right? Hexes are a bit faster to read (imho).
What we see here is a 12-byte header consisting of a 4-byte signature “DIRC” which stands for DirCache.
There’s also a 4-byte version number “2” (0x00000002), which is the current version of my git index format.

The rest is a number of index entries, the checksums we talked about during objects section.
So basically the index contains every bit of information needed to create a tree object. Not only that it can be used to compare the working tree with the currently used tree, which is a very important part of git itself (being able to see your changes, this is where it originates to). It also represents merge conflicts and helps with resolving three-way merge (a merge, where the final version is somewhere in between of two versions you had).
In final notes, I would like to recommend putting git ls-files --stage in your console and figuring out what the output tells you.

Next up I will explain in more detail what happens in git when you use some of the more popular git commands.

So what really happens when I…
1. Use git add
2. Use git commit
3. Use git branch
4. Use git checkout
5. Use git merge
6. Use git push
7. And more…

SOURCES

https://www.youtube.com/watch?v=ZDR433b0HJY
https://mincong-h.github.io/2018/04/28/git-index/
https://stackoverflow.com/questions/21309490/how-do-contents-of-git-index-evolve-during-a-merge-and-whats-in-the-index-afte
https://shafiul.github.io//gitbook/7_the_git_index.html

How to better organize your git commits

Krzysztof — Thu, 11 Apr 2019 13:58:22 +0000

Git is an admirable piece of software. It might be the best one I've seen so far in my programming career. I think it's one of those that I'm impressed the most because very often when I do learn something new it leads me to another new thing. This constant process of exploration happens very naturally to me in my daily workflow and during free-time programming workouts cause I use git as my version control system in all my projects so far and honestly I find it kinda sad that a lot of people limit their work with git to a very basic subset of commands. Amongst many programmers, younger or not, that I met personally or just happen to read about their workflow online I see a lack of interest in git besides basic commands, used for the basic workflow. It’s nothing bad, but I think it’s kind of a shame, and a wasted opportunity to learn something fun and useful.

This basic workflow with git many people limit themselves to goes as follows:

git clone and git init to get an existing project or create a new one
git branch branch_name* (or git checkout -b branch_name)
git rebase to update branches with some other branch changes
git pull to... well to pull from remote
git push to send your changes to remote
git merge - to merge...
some helpers command like status, diff, log
and obviously git add; git commit -m 'message' to stage your changes and commit them

Honestly, that usually is enough to work on a completely functional project and if none makes some mistakes you can go from nothing to a fully functioning git repository using only those commands. And if you and your team can make a new project and create a readable, sensible 'git tree' using only those commands then bravo, none made a mistake, none pushed an old branch with force, none merged merge conflicts, none rebased with the wrong branch, etc.

Anyway, I would like to encourage you to check out, and maybe use an interesting option passed into git add. I've learned about it recently and became really fond of it.

WHAT DOES add . EVEN DO?

We should probably quickly go over this, right? Without getting too much into the inner workings of git cause that's not why we are here git add . adds(duh!) changes you point to “.” indicates to everything that is currently changed but not staged from the directory we are currently in. So being in the main project directory will add all the stuff nested in it. You can also pass a certain file path) to a magical place called index. The index is this space where your changes go to, before becoming a real commit. When you commit, you take the index changes and make them into a commit. You may think of the index as a sort of checkpoint before actually doing something. In reality, git index is actually a binary file in a hidden git folder of your repo, but that is not very important right now.

WHY YOU SHOULD DIVIDE YOUR CHANGES TO MANY COMMITS

So imagine you just got a new task, and you estimate it as a task for at least 3-4 days of work. A lot of new files, lots of changes to existing ones, the task involves a lot of business changes and many logically separate components. If you are a good git worker, you will know that tasks like this should consist of many commits. Each of those commits should introduce a new business or logical change to your project. You want to push your changes in a way that will make it easy for people to type git log and then look at your last few commits and immediately know what was added from a business point of view, and why was it added this way (by commit description for example). If you commit two days of hard, intense work on a complex code that tackles a lot of problems it is very probable that this commit won't be very explicit or helpful. It might even prove confusing for other people. The way big batches of changes should be implemented is usually series of smaller commits (or at least one commit with a more explicit description if the changes are really tightly connected)

You may think this is obvious but believe me, it is not. I saw senior developers commit work worth of two weeks with a message of the date they started working on the feature. And oh my god, so many "hotfix" commits with a description of "fix2" or "fix bug_4". It's a really bad practice. I think that when we type in git log into terminal, and we scroll down a bit, we should get a good description of what has changed in our app, why it changed that way, etc. If we write our code in a way that makes it as explicit and understandable for others, why shouldn't we treat our git tree the same way? It's just plain English (or any other language) in those messages.

SO WHAT DOES add --patch DO?

When you use git add . everything is done in the blink of an eye, but git add --patch will take a while, and give you more to do (hurray, more work with git, right?). Alright, we will work on some legacy rails app, and do some mockup changes to it. Let's say we are working on a new branch, where we got a lot of different changes to do, they are all connected for the client that tasked us with the changes. They come from one ticket (task) and should all be deployed to production at the same time. But from our point of view, those changes are not that much connected. Let's see git diff of those changes.

So there's a change to a mailer that welcomes the new user to the system, there's a change in the validation of Post model, there's a change in some method of AuditLog model that probably affects its data, and finally, there's a change to PostController delete action (keep in mind those changes are just mocked, it does not matter they don't do much, what matters is that they represent different business and logical changes and can be displayed easily for me to explain).

We can have two approaches to situations like this. We could just git add .; git commit and then name the commit, and put in the description what exactly we changed. But for git messages to be easily readable they should be limited to 50 characters. Describing changes to 4 different concepts in 50 characters might prove difficult. Especially if we had some bigger changes that would take up more than 4 files.

Lets finally use add --patch

Okay so we only see one bit of change, the controller one. And we have a prompt. What is git asking us about, what is this whole hunk thing? Hunks are the bits of changed code you see separately when you use git diff. We will talk more about hunks and how git sees them soon. For now, let's just add it and go with the flow. Type y

Another hunk. Hmmm, let's not stage this one, okay? Type n

Again, we don’t want this hunk... We don’t want any other hunks added right now, we added one and we are happy with that. Type q to indicate that we should skip this and all other hunks that were not added yet. We can use d to only skip hunks from the current file.

Alright, we have 4 modified files, 1 of them is staged. Let's commit and see our logs.

Looks okay, right? Alright lets stage and commit all other changes with their own commits using git add --patch and see our final product.

Now, we have a more clear view of commits. You might ask yourself though, "well I will just plan my workflow correctly, do all business and logical changes together, commit them and move to next batch of connected changes". Sure it sounds nice and all, and if you can do that, then good for you. But reality verifies statements like this and other brave statements like "we will refactor this later". In reality, most of us when tasked with a new big feature and a limited portion of time, just jump right into it, and start building until we are finished. Create a new branch and work, work till its ready to be presented to others for code review. What's great about --patch is not only the ability to separate your work as logical batches (in the form of commits) but also a more thorough, step-by-step review of your code. Once you get the hang of it, it does not take that much time and might save you a lot of work later on. It will also help others to understand what is going on (code review by commits, not by the entire branch is easier when commits are separated nicely).