Forem: Intuit Developers

Making Accessible Links in SwiftUI

Harris Borawski — Wed, 18 Dec 2024 21:41:02 +0000

SwiftUI has streamlined Apple platform UI development, but it's not without its flaws, especially regarding accessibility. A notable concern is the inaccessibility of links within the Text view—screen readers fail to recognize these inline links as actionable elements. In this post, we'll dissect this issue, exploring its impact on user experience and providing guidelines on how developers can enhance accessibility (a11y) in their SwiftUI applications.

Problem

Let's start with some simple code to illustrate the problem. Here is a simple link within text, rendered using Markdown.

let url = URL(string: "https://developer.apple.com/")!
Text("Please review our [Terms and Conditions](\(url.absoluteString))")

This should look like this:

As you can imagine, inlining links with text is a common use case across apps. They're used for user agreements, deep-linking other pages, and simply linking to external articles.

However, when links are added like this, it is not clear to screen-readers that they are tappable. Using the Accessibility Inspector, we can see this issue in action for the code above:

"Please review our Terms and Conditions" is one label (good) with type 'text' (not good). The screen-reader will not announce this as interactive.

Solutions

Solution 1: Add a Button Trait

Adding a button trait to the Text element will ensure the screen-reader knows the element is interactive. Here’s how you could implement it:

let url = URL(string: "https://developer.apple.com/")!
Text("Please review our [Terms and Conditions](\(url.absoluteString))")
    .accessibilityAddTraits(.isButton)

However, when a text segment contains multiple links, differentiating between them becomes a challenge. The entire text block is treated as a single UI element, which isn't ideal when you want users to choose between multiple links. This solution can make the text look like a button to screen-readers, but it doesn't provide the granularity needed for multiple links within the same text.

Solution 2: Redesign the User Interface

A more robust but labor-intensive solution is to redesign the user interface so that links do not have to be appended with additional text. By isolating links from other text, you can define each link as a SwiftUI Link, which is inherently more accessible. (A Link will be presented to screen-readers as a button.)

Link("Terms and Conditions", destination: URL(string: "https://developer.apple.com/")!)

This approach maintains the visual and functional distinction of each link, improving accessibility by making each link individually selectable and actionable.

However, redesigning the user interface may not always be feasible. It can involve significant changes to the app's design and layout, which may affect the project timeline, resources, or even the overall user experience.

(For products that expect public usage, ensuring accessibility should be a priority from the outset of the design process, as retrofitting solutions often leads to compromises in user experience and functionality.)

Conclusion

Both solutions offer ways to make inline links more accessible in SwiftUI, though each comes with its own set of trade-offs. The choice between adding a button trait to the entire text block and redesigning the UI to isolate links should be made based on the specific needs of the project, the frequency and nature of the links in question, and the available resources. Of course, there are probably many other solutions as well, and I hope this article helps you on the path to exploring them!

About the Author

Koriann (like the "corian" in "coriander") South is an accomplished iOS developer with a passion for crafting seamless and accessible user interfaces. Since joining Intuit in 2022, she has played a pivotal role in the development of the company's design system, ensuring that it meets the highest standards of accessibility and usability. With a keen eye for detail and a commitment to inclusivity, Koriann's work helps ensure that Intuit's products are accessible to millions of users worldwide. In her spare time, Koriann enjoys exploring the latest advancements in technology and sharing her knowledge with the developer community.

AI Acknowledgement

This was written with the help of ChatGPT. I wrote the draft, then I used ChatGPT to rephrase my sentences, and then I rewrote the rephrasing I was unsatisified with and added a few more sentences for flavour.

Join KubeCrash 2024: Your Crash Course on Platform Engineering the Cloud Native Way

Lisa-Marie Namphy — Thu, 03 Oct 2024 16:59:20 +0000

Platform Engineers let’s unite! Calling all Kubernetes and cloud native enthusiasts to join us at KubeCrash.io!

Mark your calendars for Wednesday, October 9th, and prepare for a day packed with actionable insights and practical takeaways. We are so excited to participate in the 6th KubeCrash event with the amazing group of platform engineer software experts and open source community members.

What is KubeCrash?

Now in its sixth round, KubeCrash is a female-created and -led, open source conference and is hosted virtually just a few weeks prior to the KubeCon event by six cloud native companies who teamed up to bring you top-notch, crash courses in cloud native tech. Check out the talks from KubeCrash Spring 2024 to get a sense of what to expect.

What's New in Platform Engineering circa 2024

At KubeCrash Fall 2024 you'll learn how to navigate the hype around platform engineering solutions and explore best practices for building and integrating new tools, with deep dives into optimizing and simplifying Kubernetes for developers using AI.

Giving Back with Open Source and Deaf Kids Code

KubeCrash continues its commitment to the open source community and giving back. All KubeCrash sessions focus on open source projects, empowering you to leverage freely available, secure, and customizable technologies.

And the KubeCrash (and sponsors) commitment goes beyond tech. As with previous KubeCrash events, for every registration KubeCrash will donate $1 to Deaf Kids Code, supporting their mission of empowering deaf and hard-of-hearing children through coding education.

Why open source

Intuit believes very strongly in open source technology, and supports and contributes significantly to projects within the CNCF landscape and beyond! When it comes to enterprise-grade cloud native tooling, DevOps teams often drive the technology choices to build Kubernetes production environments. In short, open source has become critical to any modern technology stack.

KubeCrash aims to help engineering teams develop the needed skill sets to leverage these technologies in their production environments effectively. During these knowledge-sharing, virtual learning sessions, developers, reliability engineers, cloud security specialists, and platform engineers will learn directly from the maintainers of some of the most popular open source projects.

The KubeCrash Program:

Get ready for a jam-packed schedule featuring expert talks from leading companies like The New York Times, Intuit, and members of the CNCF Deaf & Hard of Hearing and Blind and Visually Impaired Working Groups.

You'll also hear directly from industry leaders and gain practical knowledge you can implement in your own work. Here's a sneak peek at some of the exciting sessions:

Opening keynote: Platform Engineering at The New York Times
Lighting talk: Introducing the CNCF's Blind and Visually Impaired Working Group
A Lightboard Talk About Platform Engineering
A Lighting Talk Introducing the CNCF Cloud Native AI Working Group
An End User Panel: Avoiding Big Mistakes - Navigating Platform Engineering Challenges
Speed Up Security: Unleash DevOps Productivity with Zero Trust Network Access
Mastering Kubernetes: Strategies for Optimal Performance and Scalability
Fail Smarter, Fix Faster: Gen AI's Role in Streamlining Build Debugging
Closing keynote: Accelerating Developer Velocity at Intuit: Our Platform Engineering Journey with GenAI and Argo

Block your calendars for these Intuit sessions

The team at Intuit was thrilled to be asked to deliver the closing keynote, and share the many learnings we’ve discovered while using GenAI to unlock developer potential. Here are some other sessions you can expect from the Intuit team:

Simplifying Kubernetes for Developers - a talk that will give our end-user perspective on using Kubernetes. Learn why abstracting networking compute infrastructure and utilizing AI-powered application autoscaling, enables developers to focus more on what they excel at: writing code.
An end user panel covering the do’s and don'ts we’ve learned while building our AI-driven platform.
Hear how Argo and other open source technology can be integrated to build your AI-native application platform, increasing developer velocity by 25%.

Don't miss out!

It’s virtual. It’s free! And it’s for a great cause! Register today and join us on October 9th for a day of learning, networking, and staying ahead of the curve in platform engineering.

See you there!

Examining Approaches and Patterns for Debuggability: Ephemeral Containers and Argo Workflows

Anusha Ragunathan — Tue, 01 Oct 2024 21:25:00 +0000

This blog is co-authored by Anusha Ragunathan , Kevin Downey and Jason Johl and is part of a series by the Intuit platform engineering team examining approaches and patterns for debuggability:

Intuit’s scale is vast, operating over 325+ Kubernetes clusters, encompassing 28,000 namespaces that serve around 2,500 production services and a multitude of pre-production services. This colossal infrastructure supports a development community of 8,000 engineers across 1,000 development teams. Consequently, debugging distributed systems at such a scale is a non-trivial task.

In this blog, we will explore the journey behind building a “Debuggability Paved Road” at Intuit. Paved Roads are a concept in the platform engineering space that define the common set of standardized tools, practices, and processes to streamline development workflows. Our endeavor was to create a standardized debugging experience across our development ecosystem to improve developer efficiency and reduce mean time to resolution (MTTR).

We’ll delve into the background of our infrastructure, the challenges we faced, and the two solutions we implemented:

Interactive debugging using ephemeral containers
One-click debugging with Argo Workflows

Finally, we’ll share our key learnings and takeaways for others looking to tackle similar challenges.

The problem: bugs

Everyone, look under your seat

With a development community as extensive as ours, one thing remains constant — bugs. These can range from esoteric 5xx errors to transient timeouts, and debugging distributed systems is notoriously challenging. Traditional methods like metrics, logs, and distributed traces often fall short when deeper debugging and observability are required. Whether it’s running a memory profiler, capturing network packets, or deploying custom debugging tools, friction in the debugging process increases MTTR (mean time to resolution) and can negatively impact business-critical applications and end customers.

We identified three primary challenges impeding efficient debugging at Intuit:

Debug Tooling Setup: Most developers have a set of favorite debugging tools, but security best practices harden app container images, preventing direct access to shells, package managers, or any sort of debug tooling.
Abstraction: Platform abstractions, although simplifying deployment, abstract away Kubernetes complexities, posing a challenge when developers need to debug their applications.
Fragmented Expertise: Debugging expertise was siloed, with experts in Java, networking, or Linux not cross-pollinating their knowledge across teams, causing inconsistent and prolonged issue resolution timelines.

To address these challenges, we aimed to create a Debuggability Paved Road, extending the concept of a developer paved road — which usually focuses on best practices and guidelines for development and deployment. First we tackled the problem of creating an automated workflow using a built-in feature of Kubernetes — ephemeral containers.

Interactive Debugging Using Ephemeral Containers

Ephemeral containers were introduced in Kubernetes 1.23 as a beta feature and became generally available in version 1.25. Unlike traditional containers, ephemeral containers can be added to a running pod for debugging purposes without restarting it. They share the same process, IPC (inter-process communication), and UTS (UNIX time-sharing) namespaces with the main application container, making them ideal for diagnostics.

Ephemeral containers were added to Kubernetes in 1.23 and onwards.

We used ephemeral containers to develop a workflow that allows service personas to easily launch a debug shell by selecting their workspace, region, and pod name. Upon clicking “Connect,” an ephemeral container is launched in the pod, targeting the main application container, thereby enabling deep inspection.

One-click debugging experience to spin up a debug shell for any service at Intuit

Here’s an example workflow and how it works in the backend:

User Request: The service persona requests a debug shell specifying the namespace, region, and pod name through the UI.
Multi-Cluster Orchestrator: Our orchestrator receives this request, authenticates it, and uses Kubernetes API (client-go) to launch an ephemeral container in the targeted pod.
Bidirectional Streaming: A secure HTTP connection is upgraded to a WebSocket, providing the user with a debug shell. All interactions are streamed back and forth between the debug shell and the ephemeral container.

Debug shell request and response flow

We also ensured our debug container image included a comprehensive toolbox, from general Linux debugging tools to language-specific packages. Security measures, such as session timeouts, RBAC controls, OPA policies, and thorough auditing, further fortify our solution.

One surprise we found is that the debug shell is not just used during incidents, but also during service onboarding. Use cases like testing database connectivity from services, checking secrets configuration, and verifying connectivity to itself and downstream services are a few of the many popular ways we have seen it used by our developer community.

Once we had our debug shell workflow defined, we tackled the next problem: a workflow engine for language-specific debugging.

One-Click Debugging with Argo Workflows

We knew there was a universal need for developers to debug running code, but often the specific debugging techniques and tools varied quite a bit depending on the language and framework. Intuit primarily uses Java Spring for web services, though we often see services in Python, Golang, and others. We knew we needed to customize the debuggability experience per language to provide a good user experience. For purposes of this blog, we will take examples from our Java debugging workflows, but we have built debugging workflows for Python and Golang as well. Let us know in the comments if you’re interested in learning more about those workflows!

For our Java developers, we implemented the ability to take thread or heap dumps via a simplified UI, select the target environment and pod, and trigger a workflow. The workflow captures the required information, sanitizes it to mask any sensitive data, and uploads it to S3 for download. This approach automates complex debugging actions and provides auditable, downloadable artifacts for further analysis.

A debug workflow involves:

Triggering a Workflow: From the UI, the developer selects an action (e.g., take a thread dump) and the relevant pod.
Multi-Cluster Orchestrator: The orchestrator determines the target cluster and namespace for the relevant pod and prepares to launch an Argo workflow in a privileged namespace of the target cluster.
Executing the Workflow: The workflow is launched in the debug namespace and captures the required data by hitting the application’s Java Spring Boot actuator endpoints.
Sanitizing and Uploading: The payload is sanitized and uploaded to S3. The developer gets a pre-signed URL to download the artifact.
Analyzing the Data: The developer can analyze the downloaded thread or heap dump using their preferred tools, such as yCrash.

Debugging workflow architecture

With the combination of the ephemeral containers for debug shells and ability to spin up these workflows on demand with Argo Workflows, we were able to realize our goal of a consistent Debuggability Paved Road for all Intuit services. From our development portal, any Intuit engineer at any level of expertise can debug their service without needing to set up complex local debug tooling.

Debuggability UI that an Intuit engineer sees for debugging a Java service. They can take a thread or heap dump of the application with one simple click.

Finally, we also wanted to ensure the stability of the underlying infrastructure during debugging. By incorporating managed endpoints and network policies, we ensure that debugging actions do not compromise the integrity and performance of the application. For example, this helps guard against crashes in the case of taking a heap dump during a memory leak or otherwise overloaded applications. Intuit engineers can debug without fear of negatively impacting the underlying infrastructure.

Takeaways and Next Steps

The journey of building a Debuggability Paved Road at Intuit has been enlightening and full of learnings. By leveraging ephemeral containers and Argo Workflows, we’ve managed to enhance our debugging capabilities, reduce MTTR, and boost developer productivity. We came away with a few larger takeaways:

Enhanced Developer Velocity and Reduced MTTR: Streamlining debugging actions with tools like ephemeral containers and automated workflows accelerates issue resolution.
Automated and Secure Sensitive Data Access: Implement efficient audit mechanisms, RBAC controls, and session management to protect sensitive debugging data.
Facilitate Seamless Collaboration: Provide consistent, auditable workflows that enable all team members to contribute to debugging efforts effectively.
Democratize Tooling: Ensure all developers have access to the tools needed to resolve issues, breaking down silos of expertise.

By establishing a Debuggability Paved Road, we’ve been able to uphold our commitment to improving developer experiences, maintaining security, and ensuring the reliability of our services. We hope our experience and solutions inspire others to create more efficient and secure debugging processes for other organizations facing similar challenges in debugging distributed systems at scale.

Stay tuned here for more updates on our Intuit development platform journey!

Intuit’s Debuggability Paved Road: A Look at Ephemeral Containers and Argo Workflows was originally published in Intuit Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Migrating CI/CD from Jenkins to Argo

Bertrand — Tue, 17 Sep 2024 18:41:37 +0000

Bertrand Quenin is a Staff Software Engineer at Intuit and Caelan Urquhart is the co-founder of Pipekit. This article is based on a presentation Caelan and Bertrand gave on November 6, 2023 at ArgoCon North America 2023. You can watch the entire session video here. For more information on this session, you can read the event details or download the slide deck.

This post is the first in a two-part series that looks at scaling cloud-native CI/CD with Jenkins and Argo. In this first post, we focus on Argo Workflows and the CI side of the pipeline. In Part Two, we’ll look at how to use Argo CD for the CD side of the pipeline.

Here in Part One, we will:

Help you understand the challenges of running Jenkins on top of Kubernetes at scale.
Show you how to use Argo Workflow alongside Argo CD to run your CI/CD pipelines. We'll cover Jenkins and Argo Workflows, see how they map, and also look briefly at an example to show the difference between the two.
Look at key considerations if you decide to migrate from Jenkins to Argo Workflows.

Let’s start by introducing Intuit. The goal for Intuit is to help and empower our customers to make the best financial decisions using our AI-driven platform. What does CI/CD at Intuit look like? We are currently running Jenkins on top of Kubernetes at scale. This means:

We have nearly 6,000 developers running 100,000 build jobs daily.
To support this, we run a Kubernetes cluster with about 150 nodes, on which are 200 Jenkins controllers.
We’re ranging between 1,000 to 1,500 build agents at any given point in time to run those builds.

The Challenges of Running Jenkins on Kubernetes at Scale

At Intuit

Working with Jenkins at Intuit has been successful but also has its challenges. One of the most common complaints that we get is that it’s hard to figure out what's going on when your build fails. The UI is not very easy to use, and it can be slow. Definitely, improvements can be made on the user experience side.

What about operational considerations, such as high availability and disaster recovery? We use the open-source version of Jenkins, which doesn't come with those features built in. So, we had to implement our own. Unfortunately, for the big Jenkins servers, it can take up to an hour to fail over one Jenkins to another region. This definitely doesn’t meet our SLAs.

There is no unified control plane with Jenkins. We’re running about 200 Jenkins servers. Even though we’ve automated as much as possible, what happens every time we need to roll out a new Jenkins upgrade or a new plugin upgrade? It's a tedious task because we have 200 servers that we need to take care of like pets.

When it comes to cost and efficiency, Jenkins is not a cloud-native product. So, to run on top of Kubernetes, the execution model that was adopted was to have one pod per build, and this pod will have multiple containers. However, this pod and its containers will run for the whole duration of your build. This means that when the build is idle—such as waiting for user input to proceed to the next stage—the pod and its containers continue running, wasting cluster resources.

At Pipekit

At Pipekit, as a startup, we face similar challenges. As a startup, our focus is on having a lean and adaptable CI approach. Since Pipekit is a control plane for Argo Workflows, our value proposition is delivering a centralized place for customers to manage their workflows and any tools or integrations that they plug into those workflows. We manage multi-cluster workflows and even integrate with several SSO providers for RBAC.

As we've shipped more and more integrations and features, our CI quickly expanded. We wanted lean and adaptable CI pipelines. We wanted the ability to iterate and remix pipelines easily. We wanted to autoscale to minimize our costs as a startup. From a maintenance standpoint, we wanted a CI approach that was a bit more “set it and forget it”, reducing the amount of work we were doing to tune our Jenkins pipelines. Finally, since we were going to deploy with Argo CD, we wanted a tool that would easily integrate with rollouts and events.

With Jenkins, the challenges at Pipekit were similar to those at Intuit.

Our builds were running too long, with CI and test pipelines taking quite a while for each PR, slowing down our team. We wanted to get PRs reviewed faster.

Also, because we're running all our containers in a pod, this limited the size of some of our pipelines and wasted cloud resources. We didn't feel like we could completely leverage spot nodes to drive down costs.

Finally, although getting started with plugins was easy, the maintenance costs increased over time. Whenever we ran into an issue with our pipeline, we had to deal with trying to figure out which plugin caused it or whether a plugin update had a security vulnerability. All of these complexities started to add up.

As a team, we were already using Argo Workflows for data processing and other infrastructure automation. So, we asked: What could we accomplish by using Argo Workflows as our CI engine?

Why Argo Workflows for CI

At Pipekit, we took a hard look at what Argo has to offer and what we needed.

First, the big benefits stemmed from running each step in a pod—by default. That unlocked some downstream benefits, like dynamically provisioning resources for each of the steps in our pipeline. This was a big win for us. We could get more granular with each pipeline step, provisioning the right resources and autoscaling it down once it's done. If a build is waiting on someone for approval, we can just spin that down until their approval is there, and then spin up a pod to complete the pipeline.

The other significant benefit we had with Argo was parallelism by default. We could define dependencies whenever they exist throughout the pipeline, and Argo would automatically run steps that don't have dependencies and run in parallel. That helped us speed up our pipelines without much effort. If we were to do this in Jenkins, we would have to be a bit more prescriptive about where we would use parallelism; and if you change your mind about that down the road, you end up with some tech debt you’ll need to refactor. With Argo, we just declare dependencies whenever we see them, and Argo runs it as it sees fit.

On the maintenance side, Argo was lightweight to deploy as you had another Kubernetes resource on our cluster. Without so many plugin dependencies, it was a lot easier to maintain.

Of course, being in the Argo ecosystem was a benefit, as we wanted to transition seamlessly into deployment or running rollouts.

Finally, not everybody on the Pipekit team was familiar with Groovy and writing Jenkins pipelines. So, it helped to have a tool that we could write with YAML. Also, Python developers could use the Hera SDK to spin up CI.

Migration Considerations

This brings us to consider the pros and cons of Jenkins and Argo Workflows for CI.

We’d still like to give Jenkins its due. It's a 10-year tool, and the community is really strong. There are a lot of great resources out there, so getting questions answered can be very quick. Argo has a strong community now, but there's still not as much of that documentation online.

From a UI/UX standpoint, Jenkins is great. It’s built for a CI experience. We were used to some of those primitives, whereas Argo Workflows is more generic. We were also using Argo Workflows for data and other infrastructure automation. If we were going to migrate, we would encounter a UX difference.

For us at Pipekit, we felt like migrating to Argo led to great autoscaling and parallelism benefits. Granted, we needed to think about how we would pass objects between steps in our pipelines. However, that extra effort in the beginning—to figure that out by using volumes or artifacts—ends up benefiting you, as you can achieve better scale and efficiency in the pipeline.

Mapping between Jenkins and Argo Workflows

Before we dive into an example, let’s briefly cover how a Jenkins pipeline maps to an Argo Workflows pipeline.

To start, we have the Jenkins file (Groovy) that maps to the Argo Workflow definition, which is either YAML or Python, if you use one of the SDKs like Hera.

A step in Jenkins maps to a task in Argo.

A stage in Jenkins maps to a template in Argo. Templates come in different flavors:

The DAG template, which is the most popular.
A steps template, which declares a sequence of linear steps
A script template, where you can pass a quick testing script in Python to be run as part of a step in the pipeline.

The shared library in Jenkins maps well to what's called WorkflowTemplate in Argo Workflows. With this, we can parameterize and remix our pipelines better than we could with Jenkins.

With Jenkins plugins, there isn’t much of a one-to-one mapping to Argo. Yes, there are Argo Workflows plugins to be aware of, but they're not built like Jenkins plugins. Argo Workflows does have exit handlers, which we can use to integrate with third-party tools.

A CI/CD Pipeline Example

Now, let’s demonstrate what a standard CI-CD pipeline can look like with Jenkins and Argo Workflows.

Our example is fairly straightforward. It starts with building a container. Then, you want to publish that container in your container registry. That stage has three steps which can run in parallel

Publish test coverage
Run security scans on the container
Perform static code analysis

After these are completed, this is where you usually want to deploy. At Intuit, we like to keep track of our deployments. So, we would first create a Jira ticket for that deployment. Then, we would deploy the new container using Argo CD. If that is successful, then we close the Jira ticket.

What does this look like concretely in Jenkins, and then in an Argo Workflow? Let’s compare by looking at some code snippets from this repository.

The Jenkinsfile

We’ll start by looking at this Jenkinsfile. The first line references the shared library that we're going to use along that pipeline.

@Library(value = 'cicd-shared-lib', changelog = false) _

In the first section here, we instruct Jenkins where to run the agent. In this case, it's a Kubernetes agent, and this is typically where you would define your pod specification. This is related to what we mentioned above, in which you would have multiple containers running for the whole duration of your build.

agent {
    kubernetes {
        yamlFile 'KubernetesPods.yaml'
    }
}

Through the rest of the Jenkinsfile, we define our stages, and stages can be nested. Notice that there's no need to specify a git clone or git checkout—it's already part of your Jenkins pipeline. It's been set up for you.

In our stage steps, we have functions such as podmanBuild or podmanMount. These functions will be defined in the shared library.

steps {
    container('podman') {
        podmanBuild("--rm=false --build-arg=\"build=${env.BUILD_URL}\" --build-arg appVersion=${config.version} -t ${config.image_full_name} .")
        podmanMount(podmanFindImage([image: 'build', build: env.BUILD_URL]), {steps,mount ->
            sh(label: 'copy outputs to workspace', script: "cp -r ${mount}/usr/src/app/target ${env.WORKSPACE}")
        })
    }
}

In the first stage (lines 12-19), we are basically building the container. We invoke the podmanBuild and podmanMount functions. The podmanMount call (line 16) is just to extract the test coverage. Something that is nice with Jenkins is that all the files you have in your workspace are available to any of the steps of your pipeline. These files are going to be reused later.

In the next stage, (lines 23-30), we publish our image, again using podman.

Finally, we have a “Container Checks” stage (lines 32-58). In the case of Jenkins, if you want to run something in parallel, it must be explicit. It looks like this:

stage('Container Checks') {
    parallel {
        stage('Report Coverage & Unit Test Results') {
            steps {
                junit '**/surefire-reports/**/*.xml'
                jacoco()
                codeCov(config)
            }
        }

        stage('Security Scan') {
            steps {
                container('cpd2') {
                    intuitCPD2Podman(config, "-i ${config.image_full_name} --buildfile Dockerfile")
                }
             }
         }

         stage('Static Code Analysis') {
             steps {
                 container('test') {
                     echo 'Running static Code analysis: from JenkinsFile'
                     reportSonarQube(config)
                 }
             }
         }
     }
 }

We have a parallel section where we will actually reuse the test coverage that we have extracted from the podman call in line 16. They're still in the workspace, and we can use them here. Then, we run some security scans, and then some static code analysis.

Finally, we have the deployment stage (lines 63-85) where we start by creating a Jira ticket. Then, we deploy using Argo CD.

`stage('Deploy') {
    steps {
        container('cdtools') {
            gitOpsDeploy(config, 'qa-usw2', config.image_full_name)
        }
    }
}`

We use a gitOpsDeploy function, which uses Argo CD under the hood, on our QA environment using the new image name.

Once the deployment is completed, we close the Jira ticket.

The Argo Workflow

For the Argo Workflow, let’s look at argo-workflow.yaml, breaking it out into a few sections.

First, we have all of the arguments that we want to pass through the workflow (starting at lines 7-20), including the Git branch, our container tag, Jira ticket number, etc.

`parameters:
  - name: app_repo
    value: ""
  - name: git_branch
    value: ""
  - name: target_branch
    value: ""
  - name: is_pr
    value: ""
  - name: container_tag
    value: ""
  - name: jira_ticket_number
    value: ""`

Next, we have the volume claim (lines 24-32), since we need to be a bit more intentional about setting up object passing with our workflow. We define the directory on the cluster where we’ll store objects as we pass between steps. Then, we set the access mode here to ReadWriteMany so that we can enable parallel read-writes throughout the workflow and achieve a higher level of parallelism as we run our pipeline.

`volumeClaimTemplates:
  - metadata:
      name: workdir
    spec:
      accessModes: [ "ReadWriteMany" ]
      storageClassName: nfs
      resources:
        requests:
          storage: 1Gi`

When it comes to the actual pipeline and how this maps compared to Jenkins, this is the templates section (lines 35-115), where our DAG lives and the pipeline is defined. Here, we set up a DAG template with several tasks.

We use what's called the templateRef to reference WorkflowTemplates. If we've applied all those WorkflowTemplates to our cluster, workflows will automatically reference those. We have a directory with our Argo Workflow templates, where we have everything defined, and we simply reference that in our workflow manifest in this definition. So, for example, in our DAG, we define a git-checkouttask and a get-git-info task to get our SHAs.

`- name: git-checkout
  templateRef:
    name: git-checkout
    template: main
- name: get-git-info
  templateRef:
    name: get-git
    template: main`

These each reference a templateRef, which we have defined in our Argo Workflow templates directory. For example, the get-git templateRef is defined in this workflow template. This approach makes it much easier to iterate on a new pipeline by just referring to the workflow templates and passing in different parameters depending on what that pipeline needs.

Moving on to the container-build task, we see that it's also using a workflow template, and it depends on the git-checkout and get-git-info tasks to run. That's how we declare the shape of the pipeline and the order.

`- name: container-build
  arguments:
    parameters:
      - name: container_tag
        value: "{{workflow.parameters.container_tag}}-{{tasks.get-git.outputs.parameters.release-version}}"
      - name: container_image
        value: "pipekit-internal/foo"
      - name: dockerfile
        value: "Dockerfile"
      - name: path
        value: "/"
      templateRef:
        name: container-build
        template: main
      depends: git-checkout && get-git-info`

If a task doesn’t use that depends key, then it'll just run in parallel automatically.

Next, we run our unit tests (lines 62-65), container scan (lines 67-71), and code analysis (lines 72-87). All of these tasks ultimately depend on git-checkout and get-git-info.

Finally, we create the Jira ticket (lines 88-92). Similar to previous tasks, we have an update-jira workflow template. In that template, we define how we want to update Jira, pass in parameters (for opening, updating, closing, etc.), and use that template throughout the pipeline whenever it's needed.

Then, we have our deployment task.

`- name: deploy-application-preprod
  templateRef:
    name: deploy-application
    template: deploy-application
  arguments:
    parameters:
      - name: app_type
        value: "preprod"
      - name: gh_tag
        value: "{{tasks.get-git.outputs.parameters.release-version}}"
  depends: create-jira-ticket
  when: "{{workflow.parameters.is_pr}} == true && {{workflow.parameters.target_branch}} == master"`

This, of course, points to our deploy-application workflow template. We pass in our arguments here for Argo CD to use to then run the deploy. That's where we effectively have a nice, seamless integration from Argo Workflows to Argo CD in our deployment.

Lastly, we wrap it up with updating JIRA again at the end (lines 105-115).

Optional: Workflow metrics and Prometheus
We also have the optional ability to add in a simple, native way to emit Prometheus metrics from our Argo Workflow. We've done that (in lines 117-157) by adding a duration metric for how long our pipeline's running, as well as counters for successful and failed runs.

Final Tips and Wrap-up

As a reminder, take a look at the GitHub repository, which has the workflow templates and structure, so you can see how Argo Workflows works for this use case. You can also check out a working example of the code to run it locally or even on your cluster.

Migrating your Jenkins pipelines to Argo might seem a bit daunting, but have no fear! You don’t have to take the approach of a full weekend with no sleep to make it happen. We recommend taking it in a piecemeal approach.

At Pipekit, our approach was first to just trigger Argo jobs with Jenkins, not completely migrating off Jenkins all at once. This enabled us to get a feel for how it would work, how stable it would be, and get that buy-in to then have confidence that we can move over larger pipelines.

When we did start moving over, we started with some of the simpler tasks and pipelines. That made it easier to figure out how we would want to migrate everything else—like our complex Jenkins pipelines.

Other tips to use when you're migrating include:

Adopt workflow templates and parameterization from the very beginning.
Think of each step as something that you will reuse down the road, and this will enable you to accelerate your migration as you get more familiar with Argo Workflows.
Don't forget to tap into the Argo community.

At Pipekit, we also have a virtual cluster where you can set up a pipeline of your own with pre-built examples. So, you won’t need to worry about configuring your local cluster or anything like that. Just spin up a virtual cluster and run that on your own.

Intuit loves open source and is a major contributor to Argo CD and Argo Rollouts, We encourage you to check out those projects as well.

Building an AI-Powered, Paved Road Platform with Cloud-Native Open source technologies

Avni Sharma — Mon, 09 Sep 2024 23:47:03 +0000

Todd Ekenstam is a Principal Software Engineer at Intuit, and Avni Sharma is a Product Manager at Intuit. This article is based on Todd and Avni's March 19, 2024 presentation at KubeCon + CloudNativeCon Europe 2024. You can watch the entire session video here. You can read the event details or download the slide deck for more information on this session.

Intuit is a global fintech company that builds many financial products like QuickBooks and TurboTax. We’re an AI-driven expert platform. How much so? To give one metric, we make more than 40 million AIOps inferences per day.

We also have a huge Kubernetes platform, with approximately 2,500 production services and even more pre-prod services. We have approximately 315 Kubernetes clusters and more than 16,000 namespaces.

Our developer community is large, too, with around 1,000 teams and 7,000 developers at Intuit working on end-user products. The service developer is among the personas who deal with the platform on a day-to-day basis. Service developers build the app logic for the product, that is then shipped to the end user. They’re focused on code and shipping fast.

Then, we have the platform persona, or who we call platform experts.

Challenges for Developers on the Platform

The overarching goal of platform engineering is to drive developer autonomy. Platform engineers focus on enabling service developers by providing capabilities through several interfaces. For example, if a developer needs a database, they should be able to access one—whether a Node.js developer or a database administrator.

Using these capabilities should be frictionless and easy. If a developer wants to deploy or manage an application on Kubernetes, they shouldn’t need to know the nitty-gritty of the platform and the infrastructure. They should be able to do it seamlessly.

But what are the challenges that service developers face today?

Steep learning curve

Our developers often deal with many Kubernetes internals and APIs, as well as infra-related configurations, on a daily basis. When something is misconfigured, they need more troubleshooting help.

Local development with dependencies

The second friction point revolves around the developer experience. Developers need help managing environments, local testing, and a lengthy onboarding process.

Tech refreshes need migrations

The third challenge concerns tech refreshes that require migration. For example, if we upgrade Kubernetes or replace the CloudWatch agent with Fluentd, we must migrate deprecated APIs. These kinds of migrations require support from the service developer team.

Consider a sample workflow our service developer goes through on our internal developer portal.

First, the developer creates an asset on our dev portal. An asset is the atomic form of deployment on our Kubernetes layer. Next, the developer develops and deploys the app.

After that, they configure specific Kubernetes primitives in their deployment repo—like PDBs, HPA configuration, or Argo rollout analysis templates. They might be onboarded to an API gateway to expose their app to the internet and onboarded to a service mesh for configuring rate limiting. Next, they have end-to-end testing of their application.

Finally, if they have any performance tests to run, they perform load testing by configuring their min-max HPA. And, of course, this is all intertwined with perpetual platform migrations—quarterly—to stay up to date.

This kind of workflow can undoubtedly drive your service developer crazy. They must focus on the business logic and deal with infrastructure concerns.

The Target State

Now that we’ve examined the challenges let’s look at our target state and where we would like to be. We want to translate all these application needs into platform means. Service developers can focus on developing the code, deploying it seamlessly without knowing the platform’s nitty-gritty details, and then performing end-to-end testing.

The platform should address all other concerns.

Addressing the Challenges

To set some context, let’s describe Intuit's development platform, which we call Modern SaaS AIR. At the top, our developer portal provides the developer experience for all of our engineers and manages an application's complete lifecycle.

From there, our platform is based on these four pillars:

AI-powered app experiences
GenAI-assisted development
App-centric runtime
Smart operations

Our operational data lake supports all of this. It provides a rich data set for visibility into how all our applications are developed, deployed, and run.

IKS AIR

Let’s focus on the runtime and traffic management component we call IKS AIR. IKS is Intuit’s Kubernetes Service layer. IKS AIR is a simplified deployment and management platform for containerized applications running on Kubernetes. It provides everything an engineer needs to build, run, and scale an application. The main components of IKS AIR are:

An abstracted application-centric runtime environment
Unified traffic management
Developer-friendly debug tools

If you build and run your own platform on Kubernetes, you likely have many of these same concerns.

Application-centric runtime

The application-centric runtime relates to two main concerns: abstracting the Kubernetes details and intelligently recommending scaling solutions.

Abstraction and simplification

The application specification abstracts and simplifies the Kubernetes details (see the left side of the diagram below) and provides an app-centric specification (see the right side of the diagram below). The platform is responsible for generating the actual Kubernetes resources submitted to the clusters.

Our application spec is heavily influenced by the Open Application Model (OAM), which describes the application in terms of components that are customized or extended using traits. Components represent individual pieces of functionality, such as a web service or a worker. Traits can modify or change the behavior of a component that they're applied to.

Through this system of components and traits, developers can define what their applications require from the platform—without needing a complete understanding of how it's implemented in Kubernetes.

Let’s consider an example of this complexity: the progressive delivery solution utilizing Argo Rollouts and Numaflow, created by the platform to enable the automatic rollback of buggy code. When a new version of an application is rolled out, canary pods with that new version are first created, and then some percentage of the traffic is sent to those new pods.

Numaflow pipelines analyze the metrics of those canary pods, generating an anomaly score. If the anomaly score is low, then the rollout will continue. However, if it is high—for example, above seven or eight—then Argo Rollouts will stop the deployment and automatically revert to the prior revision.

This is an essential aspect of how we help our developers deploy confidently without knowing how to set up this complex solution.

Intelligent autoscaling

IKS AIR also automatically recommends scaling solutions for applications. To do this, the application resource sizing, such as the memory and CPU, must be determined, and unexpected events like OOMKilled or eviction must be handled.

The platform must also handle horizontal scaling to ensure the applications operate correctly at scale and with varying load levels. It needs to identify the metrics the application should scale on and the minimum and maximum number of replicas. Naturally, these are all primarily data-driven problems.

AI will significantly impact capacity planning and autoscaling, helping us to be more efficient with our computing resources. So we're building an intelligent autoscaling recommendation system to:

Reduce the burden on our developers
Help us ensure our workloads have the resources they need
Improve the efficiency of our platform

The basic underlying idea is this: We have components in the cluster handling short-window scaling operations and emitting metrics. Subsequently, these metrics are analyzed by a group of ML models that make long-window capacity and scaling recommendations. The solutions to different scaling problems are then applied back to the clusters.

Traffic management

Another big challenge we've identified for our developers is the configuration and management of network traffic.

While some applications need to use particular capabilities in our networking environment, we found that most only need to change a few common configurations. Our solution simplifies endpoint management, unifying the configuration of our API gateway and service mesh while providing graduated complexity as required. Here is an example of the traffic configuration of a service on our developer platform.

Most applications only need to configure routes and throttling. However, if required, they can toggle on Advanced Configs, which gives them access to CORS and OAuth scopes. They can edit the underlying YAML configuration for even more complex use cases.

Debug Tools

With the platform abstracted, our service developers faced another challenge: troubleshooting their services. And we know that abstraction and debuggability don’t go hand in hand. Therefore, it was critical for us to not only provide a paved path but also to service their debuggability needs.

To accomplish this, we provided an extremely developer-friendly debugging experience in the developer portal. Service developers don't need to know about Kubernetes primitives or have any historical knowledge of the platform. Our aim is to democratize debug tooling across teams, as we saw many developers already juggling many different tools. This approach will help us reduce MTTR and friction in debugging workflows.

Interactive debugging using ephemeral containers

First, we've provided our developers with an interactive debugging shell experience using ephemeral containers. Ephemeral containers are a GA feature in the Kubernetes 1.25 release. They’re ideal for running a custom-built transient container and troubleshooting the main app container in the service pod. This is great for introspection and debugging. So now you can launch a new debug container into an existing pod.

This ephemeral debug container will share the same process namespace, PID, IPC, and UTS namespaces of the target container. Given that the containers in a pod already share their network namespaces, this debug container is set up perfectly to debug issues in the app container.

This is a demo of how this looks on our development platform:

Click on the shell icon.
Select a host, which is a pod. Select a pod and click Connect.
From here, an ephemeral container will try to connect to the particular app container.
Once it attaches and a connection is established, a session is started.

In this way, we have hidden the complexity of using kubectl exec to access a pod or even get a kubeconfig. A developer can quickly use this frictionless experience to debug their service.

One-click debugging

Another feature we have provided is one-click debugging. We have used Argo Workflows for the workflow implementation, which is ideal for defining the sequence of steps needed for a debugging workflow. Specific debugging techniques are required based on the language and framework. We also want to preserve as much application context, structures, and code references as possible while debugging a service.

At Intuit, we determined that our top two languages are Java and Golang. These are some of the language-specific debugging tools that we might use:

What does this look like in our developer portal? The user interacts with the UI to take a thread or heap dump on a target pod for a Java service. A workflow is executed in the background when they hit Add to Debug List and add that specific pod or host. This will run certain steps in sequence to perform the thread dump and heap dump workflow. The developer can later download the artifact and use their preferred analysis tool. These artifacts are available for download for only 24 hours.

Key Takeaways

To conclude this post, let’s go over the key takeaways of building this paved road:

Increases developer release velocity
Streamlines the process of performing platform migrations with very little friction for service developers and their teams
Reduces the time to get a service into production, since the platform handles much of the heavy lifting, taking that burden off the developer.
Helps to reduce potential incidents caused by infrastructure misconfigurations.
Provides a better developer experience by abstracting infrastructure network connectivity and providing intelligent autoscaling for managing service availability

Getting Started with GraphQL

Lucy — Wed, 31 Jul 2024 14:00:00 +0000

Welcome to the world of GraphQL! At its core, GraphQL is both a query language for your API and a server-side runtime for executing those queries. It uses a type system you define for your data. GraphQL isn’t just another tool—it represents a new way of thinking about how data is loaded and managed in applications. If you’re coming from a background in RESTful APIs, you are in for a treat.

Most of us are familiar with companies like Meta (Facebook), who developed GraphQL, or others who have adopted GraphQL to streamline and power their data interactions. Those companies include GitHub, Wayfair, Atlassian, and Intuit.

What makes GraphQL stand out? Ultimately, GraphQL gives you the ability to request exactly the data you need—nothing more and nothing less—and to retrieve that from a single endpoint.

In this guide, we’ll cover the essentials: what GraphQL is, how it differs from REST, and how to start integrating it into your projects. We’ll walk you through this step by step. Let’s begin with the question most asked by API developers who are new to GraphQL.

How Does GraphQL Differ from REST?

When working with APIs, you're probably used to REST, where each resource, like users or products, has its own URL. This setup can lead to a lot of endpoints, and sometimes, fetching simple information requires multiple requests. For instance, imagine you need to generate a summary of an order for a customer. You might need the date the order was placed and the names of the products in that order. With REST, this would typically involve:

Sending a request to the /orders endpoint to get the order dates.
Sending another request to the /products endpoint to fetch the product names.
Possibly sending another request to the /user endpoint to confirm customer details.

In each of these calls, REST might send back much more data than you need—like product prices, descriptions, or user addresses—which you won’t use for your summary.

Selective data retrieval
With GraphQL, you avoid this inefficiency by specifying exactly what you need: just the order date and product names, nothing more. This not only cuts down on the data transferred over the network but also significantly speeds up response times, making applications quicker and more responsive.

Single endpoint approach
Unlike REST, which uses multiple endpoints, GraphQL operates through a single endpoint. This simplifies the workflow because you don’t have to manage multiple URLs; everything happens in one place.

Efficiency and performance
One of the biggest advantages of using GraphQL is its efficiency. Since you can fetch all the required data in a single request, it reduces the number of requests sent to the server. This is particularly beneficial in complex systems or mobile applications where network performance is crucial.

Strongly typed schema
GraphQL uses a strongly typed system where each operation is backed by a schema defined on the server. This schema acts as a contract between the client and the server, ensuring only valid queries are allowed. It helps catch errors early in the development process, improving the quality of your code.

Understanding the Basics of GraphQL
As you start your journey with GraphQL, it’s important to grasp its fundamental concepts. These include the ways you can query data, modify it, and define the structure of data using GraphQL’s type system. Let’s break these down:

Core operations
At its heart, GraphQL is built around three main operations:

Queries: These are used to read or fetch data. Unlike REST, where you might receive more information than you requested, GraphQL queries are designed to return exactly what you specify, no more and no less. This precision not only makes data retrieval more efficient but also easier to predict and handle in your applications.
Mutations: While queries fetch data, mutations change data. Whether you’re adding new data, modifying existing data, or deleting it, mutations are your go-to operations. They are clearly defined in the schema, so you know exactly what kind of data modification is allowed and what isn’t.
Subscriptions: Although we won't delve deeply into subscriptions here, it's useful to know they exist for real-time data updates. Like listening to live updates from a server, subscriptions allow your application to get immediate data as soon as it changes on the server.

Type system
GraphQL’s type system is like a blueprint for your data. Here’s how it works:

Schema definition: This is where you define every type of data that can be queried or mutated through your GraphQL API. It outlines the structure of both input data (what you send) and output data (what you receive).
Strong typing: Every piece of data has a type, like String, Integer, or a custom type like User or Product. This strict typing helps catch errors during development, as GraphQL will only process queries and mutations that fit the schema.
Resolvers: For every field in your schema, there's a function called a resolver that's responsible for fetching the data for that field. This means whenever a query or mutation requests a specific field, the resolver for that field runs to retrieve or update the data.

Understanding these basics sets the foundation for effectively using GraphQL in your projects. You’ll find that with a strong grasp of queries, mutations, and the type system, transitioning from traditional REST approaches becomes much smoother and more intuitive.

Setting Up a GraphQL Environment
In this section, we’ll set up a GraphQL server to handle a dataset involving users, orders, and products. This will give you a practical feel for how GraphQL manages relationships and queries. We'll use Node.js for our server setup, and we’ll take advantage of a package from Apollo Server.

Apollo GraphQL is widely recognized as a leading provider of open-source tools for GraphQL. It offers a range of user-friendly solutions, such as Apollo Server and Apollo Studio, that help developers create and manage their GraphQL applications more efficiently. It has strong community support and straightforward documentation, making it easy for anyone to start working with GraphQL.

Installation and initial configuration
Whether you’re working in an IDE or directly from the command line, getting started with GraphQL simply requires that you have Node.js and npm installed on your machine. The easiest way to do this is through nvm (Node Version Manager), which you can use to install the latest version of Node.js and npm. You can follow these installation instructions for the simple steps to do this.

With Node.js and npm installed, you’re ready to begin working with GraphQL.

Step 1: Project initialization
First, we’ll start by creating our project directory and initializing it:

~$ mkdir project && cd project
~/project$ npm init -y

Step 2: Install dependencies
Next, install Apollo Server and GraphQL:

~/project$ npm install apollo-server graphql

Step 3: Data preparation
Prepare the data model with some dummy data by creating a file called data.json.

~/project$ touch data.json

This file will contain a mock initial data set, which makes it easier to get up and running with GraphQL, as opposed to needing to set up an entire database backend. The data.json file should have the following contents:

{
  "users": [
    { "id": "1", "name": "Alice" },
    { "id": "2", "name": "Bob" }
  ],
  "orders": [
    { "id": "1", "userId": "1", "products": [{"productId": "1", "quantity": 1}, {"productId": "2", "quantity": 2}] },
    { "id": "2", "userId": "2", "products": [{"productId": "3", "quantity": 1}, {"productId": "4", "quantity": 1}] }
  ],
  "products": [
    { "id": "1", "name": "Laptop", "description": "High performance laptop", "price": 1249.99 },
    { "id": "2", "name": "Smartphone", "description": "Latest model smartphone", "price": 575.00 },
    { "id": "3", "name": "Tablet", "description": "Portable and powerful tablet", "price": 410.99 },
    { "id": "4", "name": "Headphones", "description": "Noise cancelling headphones", "price": 199.00 }
  ]
}

Server setup

Step 4: Define GraphQL schema and resolvers
In a server.js file, define our GraphQL schema (with Apollo Server, the convention is to define this in typeDefs), which includes types for User, Order, Product, and their relationships. Also, define the resolvers to specify how the data for each field in your schema is fetched from the data source.

~/project$ touch server.js

The server.js file should look like this:

// server.js
const { ApolloServer, gql } = require('apollo-server');
const data = require('./data.json');

// Type definitions define the "shape" of your data and specify which ways the data can be fetched from the GraphQL server.
const typeDefs = gql`
  type User {
    id: ID
    name: String
    orders: [Order]
  }
  type Order {
    id: ID
    userId: ID
    products: [OrderProduct]
  }
  type OrderProduct {
    productId: ID
    quantity: Int
    product: Product
  }
  type Product {
    id: ID
    name: String
    description: String
    price: Float
  }
`;

// Resolvers define the technique for fetching the types defined in the schema.
const resolvers = {
  User: {
    orders: (user) => data.orders.filter(
        order => order.userId === user.id
    ),
  },
  OrderProduct: {
    product: (orderProduct) => data.products.find(
        product => product.id === orderProduct.productId
    ),
  }
};

We’ve defined two resolvers above. Notice how in our typeDefs that a User has an array of Orders, and an Order has an array of OrderProducts. In our resolvers, we make it clear that a user’s orders are, naturally, those where the userId matches the user’s id. A similar approach applies to an Order and its OrderProducts. We use resolvers to help define the relationship between data models.

Step 5: Initialize and launch Apollo Server
Next, we initialize Apollo Server with the schema and resolvers, and then we start it up to listen for requests. Add the following to the bottom of server.js:

const server = new ApolloServer({ typeDefs, resolvers });
server.listen().then(({ url }) => {
  console.log(`🚀 Server ready at ${url}`);
});

This setup gives us a fully functional GraphQL server using Apollo. Next, we’ll dive into how to construct and execute queries to interact with this server setup.

Working with Queries
With our Apollo Server up and running, we can build out the typeDefs and resolvers to handle basic queries.

Define Query in typeDefs

Let’s add a Query type to our typeDefs.

const typeDefs = gql`
  type Query {
    users: [User]
    user(id: ID!): User
    orders: [Order]
    order(id: ID!): Order
    products: [Product]
    product(id: ID!): Product
  }
  type User {...}
  type Order {...}
  …

These definitions establish the type of data that will be returned for each given query. For example, the users query will return an array of Users, while the user query—which will take a single id—will return a single User. Of course, it would make sense that the single User returned is the one with the matching id. However, we use resolvers to make that explicit.

Add Query to resolvers

Let’s update our resolvers object to look like this:

const resolvers = {
  Query: {
    users: () => data.users,
    user: (_, { id }) => data.users.find(user => user.id === id),
    orders: () => data.orders,
    order: (_, { id }) => data.orders.find(order => order.id === id),
    products: () => data.products,
    product: (_, { id }) => data.products.find(product => product.id === id),
  },
  User: { … },
  OrderProduct: { … }
};

We see from how we define each Query in our resolvers that the users query will map to our data.users array. Meanwhile, a specific product query will search the data.products array for the product with the matching id.

Start server
Now, we’re ready to start our server. In the terminal, we execute the following command:

~/project$ node server.js
🚀 Server ready at http://localhost:4000/

Test basic queries

To interact with our GraphQL API from the command line, we can use curl to send HTTP requests to the /graphql endpoint. If you were to visit http://localhost:4000 directly in your browser, you would see the Apollo Studio web interface (more on that below).

Let's say we want to fetch the name and orders for the user with id 1. The GraphQL query in JSON format would look like this:

~$ curl \
   -X POST \
   -H "Content-Type: application/json" \
   -d '{"query": "{ users { id name } }"}' \
   http://localhost:4000/graphql

{"data":{"users":[{"id":"1","name":"Alice"},{"id":"2","name":"Bob"}]}}

In the above request, we are sending a users query, which we defined in our resolvers to be “all the users in the data.users array.” Based on the User schema defined in typeDefs, we can ask for an id, name, and orders for each User. However, in GraphQL, we get to decide how much information we want to receive. In the case of this request, we only want the id and name for each User.

Let’s send another request. This time, we will retrieve all the orders for a specific User, and for each Order, we also want to fetch certain information about the associated OrderProducts.

~$ curl \
   -X POST \
   -H "Content-Type: application/json" \
   -d '{"query": "{ user(id:1) { orders { id products { quantity product { name price } } } } }"}' \
   http://localhost:4000/graphql | json_pp

{
   "data" : {
      "user" : {
         "orders" : [
            {
               "id" : "1",
               "products" : [
                  {
                     "product" : {
                        "name" : "Laptop",
                        "price" : 1249.99
                     },
                     "quantity" : 1
                  },
                  {
                     "product" : {
                        "name" : "Smartphone",
                        "price" : 575
                     },
                     "quantity" : 2
                  }
               ]
            }
         ]
      }
   }
}

Notice that both of our requests were POST requests to the same /graphql endpoint. All our requests will be POST requests, and they’ll all go to this endpoint. Our data payload notes that we are sending a query, and then we specify our query parameters as a JSON string.

Mutations and Updating Data

Mutations are essential for performing write operations in GraphQL. Unlike queries, which are used to fetch data, mutations change data—anything from creating and updating to deleting records. Let’s define some common mutations for your GraphQL API and show how to implement them.

Define Mutation in TypeDefs
First, we’ll extend our GraphQL schema to include mutation type definitions:

const typeDefs = gql`
  type Query { … }
  type Mutation {
    addUser(name: String!): User
    updateUser(id: ID!, name: String!): User
    addProduct(name: String!, description: String, price: Float!): Product
    addOrder(userId: ID!, products: [ProductInput!]!): Order
  }
  input ProductInput {
    productId: ID!
    quantity: Int!
  }
  type User { … }
  type Order { … }
  }

We’re adding four mutations here:

addUser will add a new user with the given string name.
updateUser will update the name of the user with a given id.
addProduct will add a new product with the given name, description, and price.
addOrder will add a new order for a user with the given id, but it will also need to take an array of ProductInput objects, which we also define as a combination of a productId and quantity.

Add Mutation to resolvers
Next, we define resolvers for our mutations.

const resolvers = {
  Query: { ... },
  Mutation: {
    addUser: (_, { name }) => {
      const newUser = { id: String(data.users.length + 1), name };
      data.users.push(newUser);
      return newUser;
    },
    updateUser: (_, { id, name }) => {
      const user = data.users.find(user => user.id === id);
      if (!user) return null;
      user.name = name;
      return user;
    },
    addOrder: (_, { userId, products }) => {
      const newOrder = {
        id: String(data.orders.length + 1),
        userId,
        products
      };
      data.orders.push(newOrder);
      return newOrder;
    },
    addProduct: (_, { name, description, price }) => {
      const newProduct = { id: String(data.products.length + 1), name, description, price };
      data.products.push(newProduct);
      return newProduct;
    }
  },
  User: { ... },
  OrderProduct: { ... }
};

In our resolvers, we specify how the Apollo Server should handle each of our mutation operations. This typically means creating a new object (or finding the object to be updated, in case of updateUser) and then updating data with that object.

Restart server
With our mutations defined, let’s restart our server with node server.js again.

Test mutation operations in Apollo Studio
Apollo has a convenient and feature-rich web interface called Apollo Explorer that can connect to our development environment server to make querying and testing easier. With our server started, we can visit the server URL (if you are running locally, then it will be http://localhost:4000) in our browser, and this will redirect us to Apollo Explorer. Alternatively, visit https://studio.apollographql.com/sandbox/explorer directly.

Let’s test out a few mutation operations in Apollo Explorer. For example, the following operation calls the addUser mutation to add a new user with the name Charlie, and then it returns the resulting id and name of the newly added user.

In Apollo Explorer, the operation looks like this:

What if we wanted to add a new order for Charlie? It would look like this:

Finally, here’s how we might perform an updateUser mutation:

Notice that mutation operations can be followed up with querying for more efficient operations.

A note on handling data
So far, our examples have used an in-memory object called data which started as data read in from a JSON file. This approach is great for learning and quick prototyping, but it isn’t suitable for production environments where data persistence is crucial.

In a production setting, you would typically connect your GraphQL server to a persistent database. The resolvers we've written, which currently manipulate the in-memory data object, would be adjusted to execute queries against this database instead.

Error Handling and Debugging
Handling errors and debugging in GraphQL may differ slightly from how you would do it with traditional REST APIs. Here are some tips for approaching error handling and debugging efficiently:

Check for errors: GraphQL operations typically return a list of errors in the response body. These might not be HTTP status errors but are instead part of the GraphQL response under the errors field. Therefore, you should always check for errors in the response alongside the data to ensure comprehensive error handling.
Logging: Implement logging on the server side to capture requests and errors. This can help you trace issues back to specific queries or mutations.
Validate queries: Use tools like Apollo Explorer during development to validate queries and mutations for syntax and structure before deployment.
Use clear error messages: Design your resolvers to throw errors which include clear and descriptive error messages. This helps clients understand what went wrong without the need to dig into server-side logs.

Tools and Ecosystem

When working with GraphQL—especially as you’re just starting out—you don’t need to build everything from scratch. Take advantage of the rich ecosystem of tools designed to enhance your development experience. Apollo Studio featuring Explorer is a prime example, allowing you to build and test your queries interactively within a web interface. This makes it easier to visualize and manipulate your GraphQL operations.

For an alternative in-browser GraphQL IDEs, there is also GraphiQL. For API testing and query building, Postman—traditionally known for handling REST APIs—also supports GraphQL. Additionally, GraphQL Voyager is a tool for visually exploring your GraphQL API as an interactive graph. This can be particularly useful for understanding and documenting the relationships between entities.

Conclusion

Congratulations on navigating through the fundamentals of GraphQL! By now, you should have a solid understanding of how GraphQL differs from traditional REST APIs, its efficiency in fetching and manipulating data, and the flexibility it offers through queries and mutations.

Throughout this guide, we've explored:

Setting up a GraphQL environment using Apollo Server.
Constructing and executing queries to retrieve data, demonstrating the efficiency of GraphQL in fetching exactly only the data that you need.
Implementing mutations to modify data.
Tips for error handling and debugging.
Additional tools that can help you in your GraphQL learning journey.

Continue exploring GraphQL, and spend time looking into advanced features like subscriptions. As you design APIs for projects in the future, consider whether the efficiency and power of GraphQL may be a better fit for your needs than a traditional REST API. Keep learning, keep building, and make the most of what GraphQL has to offer!

Open Source for Beginners: FAQ Guide

Lucy — Fri, 15 Mar 2024 21:18:09 +0000

Interested in contributing to open source, but not sure where to start? As a developer advocate working on Intuit Open Source, I hear this quite a lot from our community. To help you get over that cold start problem, we've compiled a beginner-friendly FAQ to jumpstart your first open source contribution.

If you prefer video, consider checking out our OSS beginners video guide as well!

Where can I find good active projects to contribute to?

firsttimersonly.com has a listing of various first-timer resources
GitHub issues search: #good-first-issue
awesome-for-beginners (categorized by language)
Nontechnical contribution resources: awesome-for-non-programmers

Our #1 tip: just do it! Don't get too stuck looking for the ideal first project, just choose something you can do and start getting your feet wet.

How do I feature OSS work on my resume or portfolio?
You can list open source contributions under the "Projects" or "Volunteering" headers on your resume. Rather than listing every contribution you've ever made, focus on the projects you're most associated with, and highlight the number of contributions you made to each of those projects.

Even if it's just one pull request, it's worth showing off your work to potential employers. An open source PR demonstrates that you're able to get up to speed with an unknown codebase quickly, a valuable skill for any technical role in the industry.

What technologies do I need to know to contribute to open source?
A working understanding of Git and GitHub is essential, but beyond that, the specific technology stack and tools you'll need will depend on the project you want to contribute to. If you're looking to deepen your expertise in a particular skill set, take a closer look at the open source libraries out there leveraging the tools or languages you’re hoping to learn, and find one that fits your learning goals.

E.g. As a frontend developer looking to improve my TypeScript fluency, I could search GitHub for good first issues tagged on projects that use TypeScript.

What happens when multiple people want to work on the same issue?
Most maintainers will typically have a "first come, first serve" policy when it comes to assigning issues to contributors. If you see an issue you'd like to tackle, make sure it's up for grabs before starting work by commenting on the issue and getting a confirmation from a maintainer on the project. It’s also common that someone may claim an issue and then not make progress on it for a while, at which point you’re welcome to step in and take over (with the maintainers’ blessing, of course).

When should I stop taking good first issues?
The beauty of open source is that there's always something new to learn and contribute to. Take good first issues as long as you feel like you need to, but don't be afraid to move on when you outgrow them.

How can I generate interest in a new open source project?
Building a strong community around your project is key to its success. Offer good first issues, participate in events and open source conferences, create clean documentation and video guides, and don't hesitate to ask for feedback from others in the field. Remember that it takes time and consistency to build community momentum, and there are entire teams and OSPOs dedicated to this specific problem, so don’t be too hard on yourself if things don’t take off immediately or if you can’t achieve the perfectly consistent cadence. Little by little, you’ll get there!

What other resources can you recommend for beginners?

Here's a good Git workflow walkthrough, or you can step through the first-contributions guide
Conventional commits

Let us know in the comments if you have any other questions, and we’ll answer as much as we can. If you’re still looking around for a project, our Intuit Open Source projects can also be a great place to start.

Happy coding, open-sourcerors!

Getting started with LLM and LangChain

Ayush — Wed, 13 Mar 2024 14:02:46 +0000

Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) and Artificial Intelligence (AI). However, building and training LLMs can be a complex and resource-intensive process, requiring significant computational power and expertise. That's where langchain comes in—LangChain provides a range of tools and features that address various complexities with building applications using LLMs, such as prompt crafting, response structuring, and model integration.

In this blog post, we will go over some 101-level information on LLMs, and then walk through a demo on how to build a very simple LLM-based application to query for information. Let's dive into the world of Large Language Models and explore the ways in which LangChain is making them accessible and affordable for everyone.

First, what are Large Language Models?

Large language models (LLMs) are a class of artificial intelligence (AI) algorithms that have been trained on massive amounts of text data, for example, text from books, articles, websites, or other written content. LLMs use this data to learn how to predict the next word or words that will occur in a sequence. Given some initial words in a sentence or paragraph, an LLM can generate a natural-sounding continuation of the text.

There are several different types of LLMs, each with its own unique architecture and training methodologies. Some of the most popular types of LLMs include:

Recurrent Neural Networks (RNNs): These neural networks are designed to handle sequential data, meaning language models with context, such as long sentences or paragraphs.
Transformers: Transformers were developed in 2017 by OpenAI models and have since become among the most popular LLM designs. Transformers are trained using a self-supervised learning learning method that enables them to analyze and encode vast amounts of text data while still maintaining the context of the information.
Generative Pre-trained Transformer 3 (GPT-3): This is one of the most advanced LLMs currently available, with 175 billion parameters as of 2021. GPT-3 is a neural network-based language model that is incredibly powerful at generating and understanding human language.

Large language models have many applications, including natural language search engines, text-to-speech systems, chatbots and automated customer service, and language translation software. As LLMs continue to improve, they are becoming increasingly useful tools for businesses in a wide range of industries.

LLMs are already starting to change the way we interact with written content, and as their capabilities continue to grow, businesses and other organizations are starting to realize the benefits of using these systems to streamline their operations. For those interested in AI’s future, large language models are an exciting area to follow, and LangChain is a great platform to learn more and build new AI skills.

What is LangChain?

LangChain offers a comprehensive framework for creating applications that leverage the power of large language models. With LangChain, users are able to develop chatbots, personal assistants, and other AI-powered tools that can perform a range of tasks such as summarization, analysis, Q&A generation, code comprehension, API interaction, and more.

The LangChain framework comes equipped with pre-designed chains for efficiently executing complex tasks. Users can customize existing chains or build entirely new ones by utilizing the vast library of components that LangChain offers.

The LangChain framework has numerous components that can be used to develop custom applications. These components include:

Prompts: Prompt templates play a crucial role in streamlining user input to ensure effective communication with a natural language model. Its primary function is to format user input in a fashion that matches the expectations of the language model. These templates are crucial in providing context and specifying inputs to help the model perform the intended task efficiently. As an illustration, a chatbot prompt template may incorporate the user's name alongside their question to enhance contextual relevance and improve the performance of the language model.

Chains: Chains are pre-designed language model components that accomplish specific tasks. These are pre-designed sets of components that can process input and generate output. Chains can be used as-is or customized to meet the specific requirements of a project. Custom chains can be designed by combining multiple components, creating powerful and flexible applications.

Agents: Agents are responsible for language model training and communication. Agents train models by determining the optimal input for the model and optimizing its output. They are also responsible for receiving user input, processing it, and providing appropriate output from the model. The LangChain Agent handles different input/output formats, language models, and APIs.

LLMs: Language models (LLMs) are at the core of LangChain's functionality. They analyze, summarize, generate Q&A, and understand code. LangChain enables users to create their own models or use pre-trained models from the LangChain library. The flexibility of LangChain allows users to train models on any type of data or use pre-trained models for any type of use case.

Callbacks: Callbacks enable developers to enhance the functionality of the LangChain components by adding customized algorithms. The callbacks can be used to modify the output generated by the language model or modify the input before input pre-processing.

Memory: Memory allows users to store data across multiple sessions, enabling LangChain to provide a more personalized experience to the users. Memory can be used to store user preferences, chat history, and interaction patterns, allowing LangChain to learn from previous interactions and adapt to user's needs.

Output Parsers: Output parsers take responsibility for organizing the response output that Language Model (LLM) generates. The parser performs some functions, such as the elimination of unwanted or irrelevant content, addition of supplementary information, or alteration of response format. As a significant component of the LangChain framework, output parsers play an essential role in maintaining outcome relevance, accuracy, and structure. It ensures that the LLM's responses are more decipherable, thus improving ease of interpretation and increases in practicality of such responses.

One of the most significant benefits that LangChain offers is the ability to access LLMs models without expending high development costs or needing extensive technical expertise. LangChain’s suite of tools deliver valuable assistance for prompt crafting, response structuring, model integration and other programming-related requirements.

Now that we’re familiar with the concepts, let’s walk through a simple example to better understand how various components work together. We’ll build a service that checks whether Wikipedia contains cat recordings (who doesn’t love a cat video?):

Installation and Setup

Let’s first configure the environment. Assuming Python is already installed in your working environment, the next step is to install the LangChain library using pip:

pip install langchain

As we are going to use the OpenAI’s language models, we will install OpenAI SDK as well:

pip install openai

Next, as a critical step in connecting to OpenAI's API, it is necessary to set the OPENAI_API_KEY as an environment variable. First, sign in to your OpenAI account and navigate to account settings. Then, head over to the View API Keys option and generate a secret key, which you will need to copy. Within your Python script, use the os module to access the dictionary of available environment variables and set the "OPENAI_API_KEY" to the secret API key you copied earlier.

import os
os.environ["OPENAI_API_KEY"] = "your-api-key-here"

Get predictions
Import the LLM wrapper:

from langchain.llms import OpenAI

We can then initialize the wrapper with any arguments.

gen_llm = OpenAI(openai_api_key=”…”, verbose=False)

Initialize the APIChain to query from the API endpoint:

from langchain.chains.api import open_meteo_docs

chain_new = APIChain.from_llm_and_api_docs(gen_llm, “https://cat-fact.herokuapp.com/facts/", verbose=True)

Now we’re ready to ask a question to the LLM:

chain_new.run(‘Does wikipedia have a cat recording?’)

You should get an output like this:

> Entering new APIChain chain…
https://cat-fact.herokuapp.com/facts/


[{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008800aac31001185ed07",”user”:”58e007480aac31001185ecef”,”text”:”Wikipedia has a recording of a cat meowing, because why not?”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–06T21:20:03.505Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008630aac31001185ed01",”user”:”58e007480aac31001185ecef”,”text”:”When cats grimace, they are usually \”taste-scenting.\” They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–07T21:20:02.903Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e00a090aac31001185ed16",”user”:”58e007480aac31001185ecef”,”text”:”Cats make more than 100 different sounds whereas dogs make around 10.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–02–11T21:20:03.745Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e009390aac31001185ed10",”user”:”58e007480aac31001185ecef”,”text”:”Most cats are lactose intolerant, and milk can cause painful stomach cramps and diarrhea. It’s best to forego the milk and just give your cat the standard: clean, cool drinking water.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–04T21:20:02.979Z”,”deleted”:false,”used”:false},{“status”:{“verified”:true,”sentCount”:1},”_id”:”58e008780aac31001185ed05",”user”:”58e007480aac31001185ecef”,”text”:”Owning a cat can reduce the risk of stroke and heart attack by a third.”,”__v”:0,”source”:”user”,”updatedAt”:”2020–08–23T20:20:01.611Z”,”type”:”cat”,”createdAt”:”2018–03–29T20:20:03.844Z”,”deleted”:false,”used”:false}]

> Finished chain.

Out[4]:
‘ Yes, Wikipedia has a recording of a cat meowing.’

Congratulations! Your basic setup of an LLM-based application using LangChain is now working successfully.

GraphQL, gqlXPath, and Enhanced Transformer

Yaron Karni — Sun, 07 Jan 2024 22:25:43 +0000

I recently worked on a project that required manipulating GraphQL documents for queries and mutations.
I discovered that certain techniques for working with JSON and XML could also be useful when working with GraphQL documents.

GraphQL documents are explicitly used for sending queries and mutations to the GraphQL service rather than primarily for data storage and exchange like JSON.

To modify a GraphQL document, the following steps need to be followed:

Traverse through the GraphQL document.
Identify the relevant GraphQL node or nodes.
Manipulate the GraphQL node or nodes as required.
Create a new GraphQL document with the manipulated node or nodes.
Pass the new GraphQL document to the GraphQL server to execute the query or mutation.

Let's exclude point (5) from our discussion, as it can be accomplished through multiple tools and code snippets.

gqlex will an advanced solution for the aforementioned statements.

gqlex will offer a new way to select GraphQL node

This article will explain how ti use gqlex to traverse a GraphQL document, select the relevant nodes, and manipulate the document according to your needs.
We'll also play with the code and elaborate on relevant use cases.

The library is part of the Intuit open-source community named gqlex: gqlex-opensource-intuit, which aims to provide a polyglot implementation for path-selection and transformation for GraphQL with advanced techniques to optimize the traversal over GraphQL document.

Traverse over GraphQL document

Every element in GraphQL, such as document, query, mutation, fragment, inline fragment, directive, etc., is derived from a node with unique attributes and behavior.

gqlex uses the observer pattern in which the GraphQL document acts as a subject, traverses the entire document and notifies relevant observers on the node and context.

This observer pattern separates the traversal over the GraphQL document from the execution part that the observer consumer code would like to perform.

Selection of Node

XML has XPath.
JSON has JSONPath.

Now ... GraphQL document has gqlXPath

gqlXPath can be used to navigate through nodes in a GraphQL document.
gqlXPath uses path expressions to select nodes or node on the GraphQL document.

Behind the scenes, the gqlXPath utilizes the traversal module
and selects the node according to the required expression.

gqlXPath syntax

gqlXPath uses path expressions to select nodes in GraphQL document. The node is selected by following a path or steps.

The most useful path expressions are listed below:

Expression	Description
//	Path prefix: Select all nodes from the root node and use a slash as a separator between path elements.
/	Path prefix: Select the first node from the root node and use a slash as a separator between path elements. The range is not supported when the first node is selected.
{x:y}/	Path prefix, Select path node(s), between a range of x path node and y path node (inclusion), use of slash as a separator between path elements. x and y are positive integers. All nodes are selected if no x and y are not set.
{:y}//	Path prefix, Select path node(s), between a range of first path node result to y, using a slash as a separator between path elements. x and y are positive integers. All nodes are selected if no x and y are not set.
{x:}/	Path prefix, Select path node(a), between range of x path node result to the end of path nodes result, use slash as a separator between path elements. x and y are integers. if no x and y are not set, select all path nodes.
{:}//	Path prefix, Select node(s) from the root node, use of slash as a separator between path elements.
...	Support of relative path "any" selection e.g. {x:y}//a/b/.../f any can be set anywhere in the gqlXPath, except at the end of the gqlXPath, You can set many any as you request, this will help you while selecting node in large GraphQL structure, so you won't be required to mention/build the entire node structure.

The library also provides an equivalent code-named SyntaxPath that provides gqlXPath expression abilities use of code, mainly used by automation code.

Transformer

The transformer provides the ability to transform (Manipulate) GraphQL document simply.
The transformer uses the abilities provided by gqlex such as: gqlXPath, SyntaxPath etc.

The gqlex provides the following transform methods:

Add Children - Add children node to selected GraphQL node or nodes
Add Sibling - Add sibling node to selected GraphQL node or nodes
Duplicate - Duplicate selected node by duplication number; multi nodes cannot be duplicated
Remove Children - Remove selected nodes or node
Update name - Update selected node names or node names; for inline fragments, it will update the typeCondition name
Update alias value - update field alias value and fragment spread alias value.

Code Play

Let's start with traversal over the GrqphQL document.

Start by creating my observer. Let's name it StringBuilderObserver.

The observer will append the GraphQL node to some StringBuilder.

This way, we achieve separation of concern:

Traverse over the GraphQL document nodes
Append node values with StringBuilder ....

public class StringBuilderObserver implements TraversalObserver {
    private final List<StringBuilderElem> stringBuilderElems = new ArrayList<>();

    private final boolean isIgnoreCollection = true;
    @Override
    public void updateNodeEntry(Node node, Node parentNode, Context context, ObserverAction observerAction) {

        String  message = "";
        DocumentElementType documentElementType = context.getDocumentElementType();
        switch (documentElementType) {

            case DOCUMENT:
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "Document", documentElementType.name());

                break;

            case DIRECTIVE:
                message = MessageFormat.format("Name : {0} ||  Type : {1}", ((Directive) node).getName(), documentElementType.name());
                break;
            case FIELD:
                Field field = (Field) node;
                message = MessageFormat.format("Name : {0} || Alias : {1} ||  Type : {2}",
                        field.getName(),
                        field.getAlias(),
                        documentElementType.name());
                break;
            case OPERATION_DEFINITION:
                message = MessageFormat.format("Name : {0} ||  Type : {1}",
                        ((OperationDefinition) node).getOperation().toString(), documentElementType.name());
                break;
            case INLINE_FRAGMENT:
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "InlineFragment",
                        documentElementType.name());

                break;
            case FRAGMENT_DEFINITION:
                message = MessageFormat.format("Name : {0} ||  Type : {1}",
                        ((FragmentDefinition) node).getName(), documentElementType.name());

                break;
            case FRAGMENT_SPREAD:
                message = MessageFormat.format("Node : {0} ||  Type : {1}", ((FragmentSpread) node).getName(), documentElementType.name());
                break;
            case VARIABLE_DEFINITION:
                message = MessageFormat.format("Name : {0} || Default Value : {1} ||  Type : {2}",
                        ((VariableDefinition) node).getName(), ((VariableDefinition) node).getDefaultValue(), documentElementType.name());

                break;
            case ARGUMENT:
                message = MessageFormat.format("Name : {0} || Value : {1} ||  Type : {2}",
                        ((Argument) node).getName(),
                        ((Argument) node).getValue(),
                        documentElementType.name());
                break;
            case ARGUMENTS:
                if(isIgnoreCollection) return;
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "Arguments", documentElementType.name());
                break;
            case SELECTION_SET:
                if(isIgnoreCollection) return;
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "SelectionSet", documentElementType.name());
                break;
            case VARIABLE_DEFINITIONS:
                if(isIgnoreCollection) return;
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "VariableDefinitions", documentElementType.name());
                break;
            case DIRECTIVES:
                if(isIgnoreCollection) return;
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "Directives", documentElementType.name());
                break;
            case DEFINITIONS:
                if(isIgnoreCollection) return;
                message = MessageFormat.format("Node : {0} ||  Type : {1}", "Definitions", documentElementType.name());

                break;
        }

        if(Strings.isNullOrEmpty(message)){
            return;
        }

        stringBuilderElems.add(new StringBuilderElem(message, context.getLevel()));

        levels.add(context.getLevel());
        //spaces++;
    }
    private final List<Integer> levels = new ArrayList<>();

    public String getGqlBrowsedPrintedString() {
        return getStringAs(true);
    }

    private String getStringAs(boolean isIdent) {
        int j=0;
        StringBuilder stringBuilder = new StringBuilder();
        for (StringBuilderElem stringBuilderElem : stringBuilderElems) {
            j++;
            String spaceStr = "";
            if( isIdent) {
                for (int i = 0; i < stringBuilderElem.getDepth(); i++) {
                    spaceStr += " ";
                }
                stringBuilder.append(spaceStr + stringBuilderElem.getName() + "\n");
            }else{
                stringBuilder.append( stringBuilderElem.getName() + (j+1<stringBuilderElems.size()? " " : "") );
            }
        }
        return stringBuilder.toString();
    }

    public String getGqlBrowsedString(){
        return getStringAs(false);
    }

//    int spaces = 0;
    @Override
    public void updateNodeExit( Node node,Node parentNode, Context context, ObserverAction observerAction) {
    }
}


GqlTraversal traversal = new GqlTraversal();

StringBuilderObserver gqlStringBuilderObserver = new StringBuilderObserver();

traversal.getGqlTraversalObservable().addObserver(gqlStringBuilderObserver);
traversal.traverse(file);

System.out.println( gqlStringBuilderObserver.getGqlBrowsedString());

After we saw how the traversal is working, let's digg with some examples of gqlXPath selection GraphQL nodes

gqlXPath defines an expression language as defined above, in addition, the expression language contains more terms to familiar with, Element Name and types abbreviations.
Why it's important: A GraphQL document is more than a structure similar to JSON; It also provides a DSL exposed by the GraphQL server and GraphQL language.
The types and the element name will assist the gqlXPath in selecting the exact node or nodes.

Element Names

element_name	Description
type=	Select element by type abbreviate
name=	Select element by name
alias=	Select element by alias name

Available types and abbreviation for _type _element_name

Type abbreviate	Description
doc	DOCUMENT
frag	FRAGMENT_DEFINITION
direc	DIRECTIVE
fld	FIELD
mutation	MUTATION_DEFINITION
query	OPERATION_DEFINITION
infrag	INLINE_FRAGMENT
var	VARIABLE_DEFINITION
arg	ARGUMENT

Let's practice the gqlXPath expression,

GraphQL Document

In this document, we have 2 nodes named: 'name', but of different types: argument and field.

query {
    Instrument(Name: "1234") {
        Reference {
            Name
            title
        }
    }
}

Select all nodes (double slash) 'name', which is an argument type //query/Instrument/name[type=arg]
Select first node (single slash) 'name', which is an argument type /query/Instrument/name[type=arg]
//query/.../name[type=arg] Same as query //query/Instrument/name[type=arg]
Select of 'name' node, which is a field under Reference, reside under Instrument, reside under query //query/Instrument/Reference/name
//query/Instrument/.../name Same as //query/Instrument/Reference/name
//.../name Same as //query/Instrument/Reference/name

GraphQl Document

query Hero($episode: Episode, $withFriends: Boolean!) {
  hero(episode: $episode) {
    name
    friends @include(if: $withFriends) {
      name
    }
    friends @include(if: $withFriends) {
      name
    }
    friends @include(if: $withFriends) {
      name
    }
    friends @include(if: $withFriends) {
      name
    }
  }
}

Select all query nodes named hero //query[name=hero]
Select first query (single slash) node named hero /query[name=hero]
Select all nodes named 'name', reside under friends node //query[name=hero]/hero/friends/name
Select all nodes named 'name' within a range of index 0 and index 2 (inclusion), reside under friends node {0:2}//query[name=hero]/hero/friends/name
Select node named 'name', reside under any node, which reside under hero node /query[name=hero]/hero/.../name
Select $withFriends variable reside directly under the root node named hero //query[name=hero]/withFriends[type=var]
Select include directive first node resides under friends, which resides under root query node named hero /query[name=hero]/hero/friends/include[type=direc]
Select the 'if' argument node, reside under the @include directive //.../include[type=direc]/if[type=arg]
Select episode variable //.../episode[type=var]

How to use gqlXPath in the code:


SelectorFacade selectorFacade = new SelectorFacade();

String queryString = Files.readString(file.toPath());

// query {  Instrument(id: "1234") }
GqlNodeContext select = selectorFacade.select(queryString, "//query/Instrument /   Reference  /");

Use of SyntaxPath

String queryString = Files.readString(file.toPath());

SyntaxBuilder gqlexBuilder = new SyntaxBuilder();

gqlexBuilder.appendQuery();
gqlexBuilder.appendField("Instrument");
gqlexBuilder.appendField("Reference");

// query {  Instrument(id: "1234") }
GqlNodeContext select = selectorFacade.select(queryString, gqlexBuilder.build());

And finally we will dwell on an example that will illustrate the use of gqlXPath node selection GraphQL manipulation (AKA transform).

Here a mutation GraphQL document

mutation CreateReviewForEpisode($ep: Episode!, $review: ReviewInput!) {
  createReview(episode: $ep, review: $review) {
    stars
    commentary
    stars
    commentary
  }
}

The following Java code will select and transform the mutation document,

String queryString = Files.readString(sourceFile.toPath());

TransformBuilder transformBuilder = new TransformBuilder();
transformBuilder.addChildrenNode("//mutation[name=CreateReviewForEpisode]/createReview/stars",new Field("child_of_stars"))
        .addSiblingNode("//mutation[name=CreateReviewForEpisode]/createReview/stars",new Field("sibling_of_stars"))
        .updateNodeName("//mutation[name=CreateReviewForEpisode]/createReview/stars","star_new_name")
        .removeNode("//mutation[name=CreateReviewForEpisode]/createReview/commentary")
        .duplicateNode("//mutation[name=CreateReviewForEpisode]/createReview/sibling_of_stars", 10);

TransformExecutor transformExecutor = new TransformExecutor(transformBuilder);

RawPayload rawPayload = new RawPayload();
rawPayload.setQueryValue(queryString);

RawPayload executeRawPayload = transformExecutor.execute(rawPayload);

Description:

add new children node named: child_of_stars under gqlXPath: //mutation[name=CreateReviewForEpisode]/createReview/stars

Add new sibling node named: sibling_of_stars under gqlXPath selected node: //mutation[name=CreateReviewForEpisode]/createReview/stars

Set new node name: star_new_name value to the selected node by gqlXPath:
//mutation[name=CreateReviewForEpisode]/createReview/stars

Remove selected node by gqlXPath //mutation[name=CreateReviewForEpisode]/createReview/commentary

Duplicate selected gqlXPath node 10 times //mutation[name=CreateReviewForEpisode]/createReview/sibling_of_stars

Use TransformBuilder to build the transform plan, with selected node use of gqlXPath and the command to execute.
The transform plan is loaded to the TransformExecutor with the GraphQL payload.

The execution will result in a new GraphQL document,

mutation CreateReviewForEpisode($ep: Episode!, $review: ReviewInput!) {
  createReview(episode: $ep, review: $review) {
    sibling_of_stars
    star_new_name {
      child_of_stars
    }
    star_new_name {
      child_of_stars
    }
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
    sibling_of_stars
  }
}

Last example, will demonstrate the manipulation of directive from include to exclude

Here the query GraphQL document,

query Hero($episode: Episode, $withFriends: Boolean!) {
  hero(episode: $episode) {
    name
    friends **@include**(if: $withFriends) {
      name
    }
  }
}

Here the code:

        String queryString = Files.readString(file.toPath());

        // query {  Instrument(id: "1234") }
        GqlNodeContext includeDirectiveNode = selectorFacade.selectSingle(queryString, "//query[name=hero]/hero/friends/include[type=direc]");

        assertNotNull(includeDirectiveNode);

        assertTrue(includeDirectiveNode.getType().equals(DocumentElementType.DIRECTIVE));
        System.out.println("\nBefore manipulation:\n\n" + queryString);

        // Node newNode = new Field("new_name");
        Node excludeDirectiveNode = TransformUtils.updateNodeName(includeDirectiveNode, "exclude");

        String newGqlValue = gqlexWriter.writeToString(excludeDirectiveNode);

        System.out.println("\nAfter manipulation:\n\n" + newGqlValue);

        GqlNodeContext excludeUpdateNode = selectorFacade.selectSingle(newGqlValue, "//query[name=hero]/hero/friends/exclude[type=direc]");

        assertTrue(excludeUpdateNode.getType().equals(DocumentElementType.DIRECTIVE));


query Hero($episode: Episode, $withFriends: Boolean!) {
  hero(episode: $episode) {
    name
    friends @exclude(if: $withFriends) {
      name
    }
  }
}

gqlex Use Cases

The gqlex can be used during base code when the developer is required to enrich the GraphQL document with more fields while querying the server for data, or during manipulation of data in the server so the code can articulate the relevant fields to manipulate in the service side.
gqlex can also be utilized during integration or E2E testing, generating synthetic GraphQL data with a high velocity and managed solution.

I elaborate the following use cases with more details:

Synthetic GraphQL document creation
Testing, mostly the integration testing part, will demand the ability to query the GraphQL server with different queries and mutations.

Of course, the developer can maintain large lists of example files to send to the server or to find and replace the
relevant string in GraphQL document, but it is a cumbersome solution, hard to maintain etc.

gqlex gives you the ability to manipulate the query or the mutation with ease and use configuration-wise to list your
plan of action (gqlXPath, Transform Commands, and Argument to execute], and execute the plan:

Configuration
  Plan
    steps
      step 1
         gqlXPath (String)
         transform_commands
           transform_command
               command
               argument_object_definition
      step n
         gqlXPath (String)
         transform_commands
           transform_command
               command
               argument_object_definition
  origin_file_to_manipulate

Configuration config = read_configuration_plan(plan_file);

config_verification()

build_plan -> ... use of TransformBuilder

new_graphql_document = execute_plan -> ... use of TransformExecuter

gqlex gives you versatility and the ability to produce synthetic GraphQL data and verify the integrity of a GraphQL service.

Articulate GraphQL document on-the-fly

Sometimes, you may need to dynamically build a query or mutation based on business logic or configuration and send it to the GraphQL server.
The gqlex library can assist with this.

The gqlex library only allows manipulation of the GraphQL skeleton file, not its creation.

The developer can create the skeleton GraphQL file, store the file in resource folder.
skeleton file, means file with structure but without field only.
with the of syntaxPath, gqlex can assemble the gqlXPath and set the plan strategy on the fly,
then use of TransformBuilder to build the plan, then use the TransformExecutor to run the plan.

Building AI-powered search using LangChain and Milvus

Ayush — Mon, 11 Dec 2023 06:11:32 +0000

This blog is co-authored by Ayush Pandey, Senior Software Engineer and Amit Kaushal, Software Manager at Intuit.

Artificial Intelligence (AI) has revolutionized the way we interact with technology, and one of the most significant applications of AI is in search. With the help of AI, search tools can surface more accurate and relevant results to users. In this blog, we will discuss how to build an AI-powered search engine using LangChain and Milvus.

Before we dive into the demo, let’s talk through some of the concepts and tools involved.

What is Langchain?

LangChain is a framework for developing applications powered by language models. Use cases include applications for document question answering, building conversational interfaces for database interactions, and much more. We believe that the most powerful and differentiated applications will not only leverage a language model, but will also be:
Data-aware: connect a language model to other sources of data
Agentic: allow a language model to interact with its environment

LangChain provides the modular components used to build such applications, which can be used standalone or combined for more complexity.

What is Milvus?

Milvus was created in 2019 with a singular goal: to store, index, and manage massive embedding vectors generated by deep neural networks and other machine learning (ML) models.

As a database specifically designed to handle queries over input vectors, it is capable of indexing vectors on a trillion scale. Unlike existing relational databases, which mainly deal with structured data following a pre-defined pattern, Milvus is designed from the bottom-up to handle embedding vectors converted from unstructured data.

Vector embeddings: why the hype?

Vector embeddings are a powerful tool for developers working with natural language processing (NLP) and ML applications. Vector embeddings are a way of representing words or phrases as vectors in a high-dimensional space, where each dimension represents a different feature of the word or phrase. This allows developers to perform complex operations on text data, such as sentiment analysis, text classification, and machine translation.

Let’s go over a simple explainer of semantic features: scientists sort different types of animals in the world into categories based on certain characteristics. For example, Birds are a type of warm-blooded vertebrate that are adapted to fly. Based on these features, we created word coordinates to represent animals based on their Type and Domestication score. These scores are called “semantic features,” which capture parts of the meanings of each word. Now that the words have corresponding numerical values, we can then plot these words as points on a graph, where the x-axis represents Type, and the y-axis represents Domestication score.

We can add new words to the plot based on their meanings. For example, where should the words "Lions" and "Parrots" go? How about "Whales"? Or "Snakes"?

There are also several libraries and tools available for developers who want to work with vector embeddings. Some popular libraries include Gensim, TensorFlow, and PyTorch. These libraries provide pre-trained models for word2vec and GloVe, as well as tools for training custom models on specific datasets.

Demo: Using a similarity search for asking questions from a wikipedia

First, let’s go through some prerequisites.

Install LangChain and Milvus on your local system:

! python -m pip install --upgrade pymilvus langchain openai tiktoken

Then, import required modules:

from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Milvus
from langchain.document_loaders import TextLoader
from langchain.document_loaders import WebBaseLoader

Next, import your OpenAI API key:

import os
import getpass

os.environ['OPENAI_API_KEY'] = "your-openai-api-key"

Then, load in a Wikipedia document (here we’re grabbing the article for Intuit QuickBooks) using WebBaseLoader client, and split it into chunks:

loader = WebBaseLoader([
   "https://en.wikipedia.org/wiki/QuickBooks",
])

docs = loader.load()
# Split the documents into smaller chunks
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(docs)

Afterwards, use OpenAIEmbeddings and store everything in a Milvus vector database.

from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vector_db = Milvus.from_documents(
   docs,
   embeddings,
   connection_args={"host": "HostName", "port": "19530"},
)

It’s time to try semantic searching! Let’s ask a question using LangChain and Milvus:

query = "What is quickbooks?"
docs = vector_db.similarity_search(query)
docs[0].page_content

Output:

'Retrieved from "https://en.wikipedia.org/w/index.php?title=QuickBooks&oldid=1155606425"\nCategories: Accounting softwareIntuit softwareHidden categories: CS1 maint: url-statusArticles with short descriptionShort description is different from WikidataUse mdy dates from March 2019Articles containing potentially dated statements from May 2014All articles containing potentially dated statements\n\n\n\n\n\n\n This page was last edited on 18 May 2023, at 23:04\xa0(UTC).\nText is available under the Creative Commons Attribution-ShareAlike License 3.0;\nadditional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.\n\n\nPrivacy policy\nAbout Wikipedia\nDisclaimers\nContact Wikipedia\nMobile view\nDevelopers\nStatistics\nCookie statement\n\n\n\n\n\n\n\n\n\n\n\nToggle limited content width'
The results above are decent, but need quite a lot of formatting help. Let’s try using load_qa_with_sources_chain to ask the questions instead for a cleaner output:

from langchain.chains.qa_with_sources import load_qa_with_sources_chain
from langchain.llms import OpenAI

chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)
query = "What is quickbooks?"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)

Output:
{'intermediate_steps': [' No relevant text.', ' QuickBooks is an accounting software package developed and marketed by Intuit. First introduced in 1983, QuickBooks products are geared mainly toward small and medium-sized businesses and offer on-premises accounting applications as well as cloud-based versions that accept business payments, manage and pay bills, and payroll functions.', " Intuit also offers a cloud service called QuickBooks Online (QBO). The user pays a monthly subscription fee rather than an upfront fee and accesses the software exclusively through a secure logon via a Web browser. QuickBooks Online is supported on Chrome, Firefox, Internet Explorer 10, Safari 6.1, and also accessible via Chrome on Android and Safari on iOS 7. Quickbooks Online offers integration with other third-party software and financial services, such as banks, payroll companies, and expense management software. QuickBooks desktop also supports a migration feature where customers can migrate their desktop data from a pro or prem SKU's to Quickbooks Online.", ' QuickBooks - Wikipedia \nInitial release, Subsequent releases, QuickBooks Online, QuickBooks Point of Sale, Add-on programs.'], 'output_text': ' QuickBooks is an accounting software package developed and marketed by Intuit. It offers on-premises accounting applications as well as cloud-based versions that accept business payments, manage and pay bills, and payroll functions. QuickBooks Online is a cloud service that offers integration with other third-party software and financial services.\nSOURCES: https://en.wikipedia.org/wiki/QuickBooks'}

Other search-related use cases using LangChain and Milvus:

E-commerce search engine: the language model can be trained on product descriptions and reviews, and the data can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant products based on user queries.
Image search engine: the language model can be trained on image captions and tags, and the images can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant images based on user queries.
Video search engine: the language model can be trained on video titles and descriptions, and the videos can be converted into vectors using Milvus. The vectors can then be indexed in Milvus, and a search interface can be built to retrieve relevant videos based on user queries.

By following the simple steps we’ve outlined here, developers can use LangChain and Milvus to build search engines for various use cases ranging from a simple document search to applications in e-commerce, image, and video search. We hope this was a helpful starter guide, please leave a comment if you have any further questions!

*References and further reading: *

Hacktoberfest 2023 @ Intuit: Maintainer Spotlight

Lucy — Wed, 25 Oct 2023 17:15:30 +0000

As Hacktoberfest 2023 comes to an end, we at Intuit are excited to send rewards to our internal and external contributors for their amazing contributions. But before we do, we want to take a moment to recognize our maintainers!

Intuit has been participating in Hacktoberfest since 2019 by offering special incentives to employees for contributing to open source projects, and this year we took it to the next level by sending rewards (cool t-shirts!) to outside contributors to Intuit libraries as well. It’s important to remember that these contributions are only possible thanks to the hard work of our maintainer teams reviewing pull requests and keeping issues up to date.

We wish we could highlight every single one of our 130+ open source projects here, but here’s a short list of featured maintainer teams and their projects:

Graph Quilt

graph-quilt is an open source Java library that provides recursive schema stitching and Apollo Federation style schema composition for GraphQL APIs. The project includes a modular set of libraries with features including a Representational State Transfer (REST) adapter, GraphQL authorization, and a reference implementation of a graph-quilt gateway. There are even more features coming soon, so keep an eye out for updates!

Maintainers Ashpak Shaikh, Carlo Aureus, and Kyle Moore released a video demo and walkthrough on the Intuit Developers YouTube channel. Check it out here:

Ashpak also recently spoke in-depth about graph-quilt at GraphQLConf 2023. See the recording of his talk here!

Numaproj

numaproj is a collection of Kubernetes-native tools for doing real-time operation data analytics. Currently it includes numaflow, a massively parallel, real-time data and stream processing engine, and numalogic, ML models and tools for real-time operational data analytics.

Maintainers Derek Wang and Vigith Maurice recently gave a talk at All Things Open walking through how to build a large scale AIOps platform with open source technologies, including numaproj tools. The recording of their talk will be live soon on the All Things Open YouTube channel, so stay tuned! Maintainer Sri Harsha Yayi also joined Derek for a presentation at the Hacktoberfest Happy Hour we hosted at Intuit’s Mountain View and San Diego campuses.

numaproj is still a young and growing project, and if you’d like to follow along on the journey, consider joining the team on Slack, or follow the numaproj blog.

Player

Player is a framework for building cross-platform dynamic experiences. The core engine is authored in TypeScript with specific adaptors to natively render on iOS, Android, and React. Start by supplying some semantic JSON content, where you can describe views, your data, validation rules, and much more. Add your own asset library to handle the rendering, and voilà, you have a full dynamic user experience.

Player’s maintainer team is one of our most active here at Intuit, and they’ve been an incredible source of open source energy, year-round. To see the library in action, check out our YouTube video featuring maintainer Adam Dierkens here:

Cello

Cello is an engine for Cloud deployments: infrastructure as code (IaC) with GitOps, with isolation from your Cloud provider, all Cloud-agnostic. It fits into any CI/CD system — Jenkins, GitHub actions, you name it. We developed Cello to help us manage our complex set of various IaCs.

Maintainer Jerome Kuptz recently gave a lightning talk demonstrating Cello’s features for a round of Hacktoberfest lightning talks we did internally at Intuit. We’ll work on getting that talk shared soon on the Intuit Developers YouTube channel, but please check out Cello’s Quick Start guide in the meantime!

Trapheus (and TrapheusAI)

Trapheus is a tool for restoring relational database service (RDS) instances in AWS without worrying about client downtime or configuration retention. It supports individual RDS snapshot as well as cluster snapshot restore operations. Modeled as a state machine, with the help of AWS step functions, Trapheus restores the RDS instance in a much faster way than the usual SQL dump preserving the same instance endpoint and configurations as before.

Maintainer Rohit Kumar recently published a blog post on Intuit’s Engineering Blog celebrating 3 years since Trapheus’ first release during Hacktoberfest 2020, and announcing the launch of TrapheusAI, a next-generation data search and analysis assistant that can overcome the challenges of traditional keyword search with powerful data analyses using natural language. See the demo here:

argoproj

How could we do an Intuit open source software maintainer feature without Argo? Argo Project is one of our largest open source projects, with a vibrant contributor, maintainer, and user community. Argo is a collection of open source tools for Kubernetes to run workflows, manage clusters, and do GitOps right. The project includes argo-workflows, argo-rollouts, argo-cd, and more.

Maintainer Michael Crenshaw gave a talk on Argo CD for the Hacktoberfest lightning talks mentioned earlier, which we’ll work on publishing soon. The Argo community is also hosting an in-person Kubernetes meetup at the Intuit Mountain View campus.

—

This was only a tiny fraction of all the open source projects we’re proud to work on here at Intuit. Hacktoberfest may be drawing to a close, but the open source work continues through the rest of the year. Whether you managed to get a contribution done this October or not, we hope to see you out there in the open source community.

A huge, endless thank you to all of our maintainers and contributors for your hard work and dedication to the open source community. There are still a few days left in October, so keep those contributions coming (see Intuit’s Hacktoberfest incentives here), and Happy Hacktoberfest!