Forem: Passenger Code Alignment Assistant

Cooeee World! Passenger 🗞️🐦

Adam Crockett 🌀 — Thu, 02 Nov 2023 08:20:56 +0000

I have written extensively about a new category of tools known as Code Alignment Assistants. Both Passenger and I are frustrated by the significant challenge of aligning developer efforts with business objectives. This often requires extensive reasoning, adherence to various agile methodologies, and a range of ceremonies—a process not all developers are comfortable with. Many of us entered the field of programming because we enjoy coding itself, not necessarily the additional processes that have become associated with it. I firmly believe that integrating an AI between the developer and their tasks can streamline this translation of business needs to developer actions, reducing the need for formal processes. I am so convinced of this idea’s potential that I have decided to establish a new organization to pursue it further.

It's FOSS tooling, it's not for profit and it's free, it will make your life better by taking all of the reasoning out of your work and replace it with plain English. Complex issue/tickets become as easy to follow as a conversation with your family.

Passenger org is now going to be keep the updates flying in.

Code Alignment Assistants are the future of your ceremonies, we already have co-pilot for your code but this is something different. Imagine going to work, developing, pointing an AI at an issue, and getting to work, as you code you get feedback about your choices versus the business requirements, scoring your efforts and how closely they address the business needs, a form of testing that once above a threshold will pass.

Peckish? Follow me for more!

(Code Alignment Assistant) Passenger gains GPT-4 accuracy and GitHub App Integration

Adam Crockett 🌀 — Mon, 30 Oct 2023 23:26:22 +0000

Pigeon Progress!

Just a quick refresher on my thoughts: I'm convinced that Passenger has the potential to transform our development approach. By enhancing BDD with AI and enabling AI to evaluate our code against the desired solution directly from our issue tracker (in this case, GitHub), we can gain unprecedented insights. This feedback loop will not only alert us to errors but also offer improvement suggestions. It's about streamlining our processes, shedding unnecessary formalities, and accelerating our pace. Most importantly, it underscores the notion of empowering humans in the development process, rather than replacing them with AI. It's a game-changer, putting humans at the forefront of innovation.

Security improvements

To prepare Passenger for enterprise-level deployment and launch, I've developed a GitHub app. This can be seamlessly integrated into your individual GitHub repositories or an entire organization. The beauty of this is that it offers fine-tuned permissions, ensuring Passenger access only what's necessary. No more reliance on Personal access tokens or individual developers; you can now grant permissions organization-wide. And guess what? We're gearing up for the next big thing - Actions! By doing all of this work we addressed a significant security concern by eliminating the need for personal access tokens.

Simply install the GitHub App (note: it's still in private beta) and set up the Passenger CLI. Input your environment variables, and you're all set to enjoy helpful pigeons!

Here is what the GitHub App description has to say: (feel free to skip if you understand what Passenger is already.)

Have you ever meticulously coded a solution, only to discover it wasn't addressing the core issue? That's where Passenger comes in.

Passenger is a Code-Alignment Assistant (CAA) the first in its class, crafted with the belief in a harmonious co-working future between AI and developers. Our tool enhances development workflows, ensuring code consistency with project issues and fostering a collaborative environment where AI and human intelligence complement each other.

Passenger establishes a bridge between GitHub issues and your codebase, providing a robust tool for developers to validate that their work aligns with documented requirements and discussions. It offers the ability to compare the current state of your source code, staged changes, or pull requests against the details outlined in GitHub issues. This ensures a clear and accurate code-to-issue alignment, aiding in maintaining consistency and adherence to project specifications.

Permissions
Read access to code, discussions, issues, metadata, and pull requests

Demo time!

Im going to let the shell do the talking a moment.
coo-cart.js, it's the wrong solution on purpose

let cart = [];

function addToCart(productId, productName, productPrice) {
  let productInCart = cart.find((product) => product.id === productId);

  if (productInCart) {
    productInCart.quantity += 1;
  } else {
    cart.push({
      id: productId,
      name: productName,
      price: productPrice,
      quantity: 1,
    });
  }

  renderCart();
}

function renderCart() {
  const cartItemsElement = document.getElementById("cartItems");
  cartItemsElement.innerHTML = "";

  let total = 0;

  for (let product of cart) {
    const listItem = document.createElement("li");
    listItem.textContent = `${product.name} (x${product.quantity}) - $${
      product.price * product.quantity
    }`;
    cartItemsElement.appendChild(listItem);

    total += product.price * product.quantity;
  }

  document.getElementById("totalPrice").textContent = total;
}

# Report on issue 1 from our github repo which has the Github App installed.
passenger score 1 ./docs/sample/src/coo-cart.js

GithubConnector
adam-cyclones/passenger

      ░░                                                                      
      ░░████░░                                                                
        ██▒▒▓▓                                                                
        ▓▓▒▒██                                                                
      ▒▒▒▒░░░░▒▒                                                              
      ▓▓░░░░    ██░░░░                                                        
      ▒▒░░░░░░░░░░▒▒▓▓██░░                                                    
        ▓▓  ░░░░▓▓▓▓▓▓▓▓▓▓                                                    
        ░░▒▒▓▓▒▒                                                              
          ░░▒▒    ░░                                                          
          ░░░░          Cooeee! Percy is working please remain calm..                             

Issue recieved... #1 "Add Customisable Pigeon Animations for Nest Notifications" (fetched from live)
Converting issue to cucumber format...
The cucumber...
""" feature
 Feature: Customizable Pigeon Animations for Nest Notifications

Background: As a user of PigeonTasks app, 
  I want to customize pigeon animations for Nest Notifications, 
  So that I can enhance my user experience and add personalized touches to the app.

Scenario: Selecting Pigeon Animation from the Library 
  Given there is a selection of pigeon-themed animations
  When I go to select an animation for my notifications
  Then I should see a range of pigeon animations (from a dancing pigeon to one in themed costumes)

Scenario: Personalizing Settings for Pigeon Animations
  Given there is 'Personalized Settings' option
  When I select specific animations for tasks coming from certain friends or for tasks with certain tags/labels
  Then those specific animations should be set according to my preference

Scenario: Previewing Pigeon Animation
  Given there is an option to preview the animation before setting it 
  When I select an animation from the library
  Then I should be able to preview it within the app

Scenario: Checking Performance post Animation Implementation 
  Given there are new pigeon animations additions
  When I use the PigeonTasks app 
  Then the app's performance should not be hindered or cause lag when notifications pop up 
"""
Comparing... './docs/sample/src/coo-cart.js'
Generating report...
{
  "accuracy": 0,
  "analysis": "The provided source code is not related to the described issue. The code is about product shopping cart operations like adding to cart and rendering the cart, which doesn't pertain to the feature described about customizable pigeon animations for Nest Notifications in a task application. Therefore, the accuracy of the source code to address the issue is 0%."
}

This is now using Chat GPT 4 and that took care of the bugs and allows for much better reports!

How would I use this? Whats next?

Im really excited to start caching some of the responses, speeding it up, improving it and maybe even adding a UI who knows, a SASS, a Pipeline? You can hope to have this as part of your test suite locally as you develop and also in a PR as a check that must pass a threshold.

Percy and Passenger away!

Passenger's Leap: Bridging Jira and Source Code with Precision

Adam Crockett 🌀 — Thu, 07 Sep 2023 21:15:02 +0000

I Wanted to share a late-night breakthrough with Passenger (unreleased yet). It's now adept at comparing live Jira tickets directly against any source code, regardless of the language. And yes, it's as powerful as it sounds.

I tested its mettle against some intentionally mismatched code. The result? Passenger gave feedback that was spot-on, almost mirroring a seasoned developer's scrutiny. Imagine integrating this level of review seamlessly into your BDD process.

I'll be diving deeper soon, but felt this was too promising not to give you all a sneak peek. Eager to hear your insights.

Match Score:

    Topic Mismatch:
        Pass/Fail: Fail
        Source: The main focus is on "PigeonPoints Accumulation Issue".
        WIP: The focus is on "Adding items to cart".
        Feedback: The WIP implementation does not match the topic from the source of truth at all.

    Scenario Mismatch:
        Pass/Fail: Fail
        Source: A single scenario focusing on accumulating PigeonPoints when tasks are completed.
        WIP: Multiple scenarios focusing on adding products to a cart.
        Feedback: The WIP implementation's scenarios are entirely unrelated to the scenario in the source of truth.

    Steps Mismatch:
        Pass/Fail: Fail
        Source: Steps revolve around installing an app, completing tasks, and observing pigeons.
        WIP: Steps revolve around product addition to a shopping cart.
        Feedback: None of the steps in the WIP implementation align with the steps from the source of truth.

    Use of Scenario Outlines and Placeholders:
        Pass/Fail: Fail (based on relevance to source of truth)
        Source: No use of placeholders (<...>) or scenario outlines.
        WIP: The WIP implementation makes use of placeholders and implies the use of scenario outlines, even if not explicitly mentioned.
        Feedback: The WIP adds complexity that wasn't present in the original scenario from the source of truth.

    Overall Relevance:
        Pass/Fail: Fail
        Feedback: The WIP implementation seems to be for an entirely different feature and doesn't relate to the source of truth in any clear way.

Overall Feedback:

The WIP implementation does not align with the source of truth. They represent different features and different domains of problems. It's crucial to revisit the requirements and ensure that the WIP implementation aligns with the intended feature to avoid potential misalignments in future development phases.

Its particularly interesting that it calls you out and tells you to go read the ticket again!

Amalgam: Roadmap to 0.1.0

Adam Crockett 🌀 — Fri, 25 Aug 2023 18:06:21 +0000

900 of you lovely people ❤️ have read about my promise of never writing tests again whilst using AI to score your solution agains for example Jira, and then it will provide feedback. Well you folks also provided great feedback thank you!

Entering production

After swearing that it would need a team and I probably won’t write much more than a prototype, I’m afraid I got a little hooked on the idea. So tonight at 8pm A repository will go live. I have another contributor I will be inviting too.

No rust

I was using rust, but I need to be productive and write faster iterations and then we will focus on performance, write version 2.0.0 in rust.

How would you use the tool

I’m going to develop the tool to work at the pull request stage, I think this will make discovering files that as a whole contribute towards solving a ticket and therefore all can be scanned, graded and your report will be delivered to you.
Therefore it is initially a pipeline tool.
I do want to make it more real-time and performant so that is where IDE plug-ins and a cli come in.

Developing ideas

It’s been said that I could even use tickets to generate code and gherkin. So yea this is now a bi directional concept and that means your scrum master could generate a starting point potentially

Right well that’s it for now. I’ll keep you posted!
🙌

Never write tests or docs again! - Amalgam could change everything.

Adam Crockett 🌀 — Thu, 17 Aug 2023 22:20:54 +0000

Amalgam is my little experimental tool written in rust and my attempt to consider the entire software development lifecycle and question it, can we do better, can we disrupt the way we develop, (yeah, I know, I hate that phrase too) but its true, and here's how.

Your code should be an echo of your tests and docs, but it often seems to be that one of the 3 is out of sync or wrong, you and your team chase the tails trying to keep it all together but like it or not, tests can be wrong, docs can be wrong and code can somehow work but still be wrong.

How can code be wrong? 🤔

It can work, but it can be wrong, in the way that the party who asked for the code via a ticket for example, was misunderstood - im no 10X dev im not super human, I do misunderstand tickets, well wrote or otherwise, so I wanted to use AI to solve this. Removing the horrific pain point, realising 2 weeks later that you have wrote the wrong solution. 🤨😭

I know of Cucumber (I may say Gherkin, I mean that), in essence its formal syntax and language for generating tests from acceptance criteria - you know, that thing that makes what you’re writing actually the thing that was needed.

Scenario: Breaker guesses a word
  Given the Maker has chosen a word
  When the Breaker makes a guess
  Then the Maker is asked to score

Because ticket writing is hard.

It is also a machine readable language and in my opinion, it is something with a barrier to entry, you need to learn but you have to commit to it to understand it let alone write it, I feel that very few teams have such time to learn it and manually write it, so enter OpenAI 🦾.

What can Amalgam and OpenAI AI do for us? 🤖🪄

cargo run -- docs/sample/src/coo-cart.js

Input:

function addToCart(productId, productName, productPrice) {
  let productInCart = cart.find((product) => product.id === productId);

  if (productInCart) {
    productInCart.quantity += 1;
  } else {
    cart.push({
      id: productId,
      name: productName,
      price: productPrice,
      quantity: 1,
    });
  }

  renderCart();
}

HTTP response:

Scenario: Add a product to the cart
    Given an empty cart
    When I add a product to the cart with id "productId", name "productName", and price "productPrice"
    Then the cart should contain the product with id "productId", name "productName", price "productPrice", and quantity 1
    And the cart should be rendered with the product details and total price

But Adam! This isn’t how we use Cucumber 🥒!! No it isn’t, it’s the reverse engineering of the traditional model, take the code and turn it into a serialised format on change. We are creating a feedback loop potentially generating and updating documentation too and creating the arse-backwards Cucumber then we can do these crazy things:

(Actually every function in this file becomes a source to generate and run tests for one, and docs for two.

But here is where it gets really saucy and new, we pass it the html page, this FAKE Jira designed to represent some source of truth

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Jira Ticket: WPTCS-001</title>
    <style>
      body {
        font-family: Arial, sans-serif;
        margin: 20px;
        background-color: #f4f5f7;
      }

      .ticket {
        background-color: #fff;
        padding: 20px;
        border-radius: 5px;
        box-shadow: 0px 0px 10px rgba(0, 0, 0, 0.1);
      }

      .header,
      .content {
        margin-bottom: 20px;
      }

      h1 {
        color: #0079bf;
        margin-bottom: 10px;
      }

      h2 {
        margin-top: 20px;
        color: #5e6c84;
      }
    </style>
  </head>

  <body>
    <div class="ticket">
      <div class="header">
        <h1>Jira Ticket: WPTCS-001</h1>
        <p>
          <strong>Epic:</strong> Nature-Themed Store Experience |
          <strong>Sprint:</strong> Sprint 5 | <strong>Priority:</strong> High
        </p>
      </div>

      <div class="content">
        <h2>Summary:</h2>
        <p>
          Develop a wood pigeon-themed cart system enhancing user experience in
          nature-themed stores.
        </p>

        <h2>Description:</h2>
        <p>
          Our objective is to introduce a cart system inspired by the aesthetics
          of a wood pigeon. The cart should be visually appealing,
          user-friendly, and resonate with our brand's commitment to nature.
        </p>

        <h2>Acceptance Criteria:</h2>
        <ul>
          <li>
            Feathered Design Aesthetics: Carts must have blue-grey and pink
            hues, handles with black striping, wood pigeon silhouette on the
            front, and signature white wing markings on the side.
          </li>
          <li>
            Cozy Nest Cart Bay: Carts stored in a "Cozy Nest Bay", allowing
            efficient stacking, utilizing recycled materials, with soft lighting
            for nighttime.
          </li>
          <li>
            Interactive Pigeon Sounds: Sensors activate cooing sounds based on
            cart interactions.
          </li>
          <li>
            QR Code Storytelling: Carts feature a QR code linking to an
            informative wood pigeon page, with engagement-based discounts.
          </li>
        </ul>

        <h2>Dependencies:</h2>
        <p>
          Collaboration with design and multimedia team for QR content.
          Consultation with hardware vendors for sound integration.
        </p>
      </div>
    </div>
  </body>
</html>

OpenAI Will be able to figure out how closely matched the code is to the criteria giving us a score by comparing it to the Gherkin.

“Does this ticket match our code?”

Outcomes, would be some sort of correctness report, run-able test for the language, documentation and guidance to improve correctness.

Difficulties

one file doesn’t describe a feature usually so how to collate that?
The length of the post limits the file sizes we can POST
Are you comfortable to POST your source?
Are you comfortable to POST your Jira?
What do you see are the problems?

Anyway what do you think?