Forem: Jobber

Removing Friction: How AI Tools Streamline Our Engineering Workflows

Sophie Dubois — Tue, 25 Nov 2025 19:58:01 +0000

Hi! I’m Sophie, a Software Engineer on the Network Venture team at Jobber. Our team spends a lot of time obsessing over ways to make Jobber simpler and easier for our customers to use. We try to remove unnecessary clicks, reduce decision fatigue, and help users get to value faster. Every week, we ask ourselves: “Where’s the friction? How do we clear it away?”

It makes sense, then, that the biggest change in how our engineering team works lately comes from applying that same philosophy to ourselves. Our developers have started using AI-powered tools to streamline how we write and test code. Tasks that used to involve half a dozen tabs now happen directly in our code editor. In short, AI is doing for us what we’ve always tried to do for our customers: cutting out the friction.

The Enemy: Context Switching

Historically, building even a small feature required juggling an overwhelming number of tools. Let’s paint a picture, shall we? We want to experiment with a new widget on our home page. Seems straightforward enough, right?

We start in Jira, where the feature requirements live, and gather all the context we need.

We’re not totally sure what the best practices are for this type of experimentation, so we switch over to Confluence and do a bit of digging through our docs.

Once we feel confident with our approach, we hop into our code editor to get started.

Halfway through, a pesky bug pops up that we can’t quite pin down. Time to jump out to Stack Overflow or Google.

A few minutes later, we realize we’ve forgotten a small but important detail from the requirements, “what’s the exact copy for the button again?”. Back to Jira we go.

By the time we’ve bounced between tickets, docs, forums, and our editor, the actual “building” feels like just one part of a much bigger juggling act.

If only there were a way to stay focused in one place, while still having all the context and answers at our fingertips...

Cursor to the Rescue

Cursor is an AI-powered code editor that keeps us in the flow. We can write code, debug issues, and pull in information without ever leaving the editor. We’ve been really excited about the efficiency gains we get with this tool so we’ve doubled down on making it even more useful by writing Jobber-specific Cursor rules and connecting it to our own Jobber MCP server.

Cursor rules are directions we give to Cursor with our own best practices baked in. Instead of just asking AI to “build a GraphQL mutation” or “write a test,” we can write rules that tell Cursor exactly how we expect those things to look at Jobber. The result is code skeletons and patterns that already follow our conventions.

Our MCP server takes it one step further, allowing us to hook Cursor directly into our Atlassian environment. That means all the best-practice documentation from Confluence and the project requirements in Jira tickets are now just a prompt away, right inside the editor. In practice, that means less time hunting for answers and more time building!

The Impact

Let’s revisit that same widget experiment scenario, but now with Cursor in the mix.

We kick things off right inside Cursor. The requirements from Jira are pulled straight into the editor with a single prompt, and we can double-check our approach by pulling in best practices from Confluence — right there beside our code.

When a bug shows up, we just drop the error into the Cursor chat and troubleshoot it together. No more bouncing between tools. Just steady, focused building.

Since we’re already in flow, why stop there? We can ask Cursor to spin up some specs for us. We have a Cursor rule for that. And while we’re at it, let’s have Cursor draft the pull request. There’s a rule for that too!

But the real impact goes beyond saving a few clicks. For engineers, it means more focus time and less mental overhead. For teams, it means less wheel spinning, faster iteration cycles, and more consistent code quality because best practices are baked right into the workflow. The small efficiencies compound, allowing us to experiment quickly, learn fast, and adapt.

The irony isn’t lost on us. As a Growth team, we focus on reducing friction for Jobber customers, and now with AI, we’re doing the same for our own workflows. In other words, we’re not just practicing what we preach, we’re coding the way we want our customers to work: simply, seamlessly, and without friction.

About Jobber

Our awesome Jobber technology teams span across Payments, Infrastructure, AI/ML, Business Workflows & Communications. We work on cutting edge & modern tech stacks using React, React Native, Ruby on Rails, & GraphQL.

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Designing simple, powerful lists 💪

Laura Erickson — Thu, 07 Mar 2024 18:45:25 +0000

At Jobber, we design and build products that contain everything field-service businesses need to run their operations effectively. With so much information to manage in our interface, it’s critical to layer data on the right surface so our customers can quickly find the information they need, and act on it to run their businesses as efficiently as possible.

We found that as our customers' businesses grew, they were outgrowing the functionality we were providing around maximizing productivity 🌱. Over time, lists in our product had been implemented with inconsistent patterns, they required navigating back and forth to complete basic actions; and were visually very busy.

We started identifying commonalities across these lists and discovered many opportunities to standardize the content hierarchy, streamline repetitive tasks with row-level actions, and simplify sorts and filters. We began rebuilding with goals to:

reduce visual clutter;
create consistent intuitive patterns; and
⚡ supercharge jobs-to-be-done;
all while improving performance.

We wanted to get to a high level of confidence in our solution before building it into the product, so when we landed on an iteration we felt good about, we developed a research plan and a fully interactive prototype to share with customers. With some adjustments, we were able to validate our simplified UI and decided to stress-test it with our customers. We built and released the first list into production for a small number of users, adjusting and expanding as we felt was necessary.

By giving our customers the right information and allowing them to act on it, we prevented having to navigate back and forth from their client list over 15,000 times just in the first month. Whenever you hear “Happy customer!!!” feedback, you know you’re on the right track.

This initiative is a great example of user-centric, design-led product development, and is one of many significant improvements we’ll be making to the product experience this year. I’m ready to push boundaries and drive towards excellence for our users 🏎. Now: onto the next simple, powerful thing!

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Behind the scenes with Jobber’s Dev Acceleration team

Tim van der Kooi — Wed, 17 Jan 2024 22:10:48 +0000

Many software companies love to obsess about customer obsession. That’s no different at Jobber: our teams are honed in on providing an easy-to-use experience for service professionals so they can focus on providing value to their customers.

But what about developer obsession? How can we do the same thing for our developers and make it easier for them to deliver value to service professionals?

At Jobber, we decided to tackle that problem with a new team called Dev Acceleration. The goal of this team is to address the bottlenecks and pain points that hinder development productivity and reduce friction with suboptimal tools or processes.

This type of team can go by many different names: dev acceleration, developer experience, or even engineering enablement. The norms around naming this team isn’t commonplace at many software companies, but the core idea is spreading.

This post will take a behind-the-scenes look at what it’s been like to establish the Dev Acceleration team at Jobber.

Know your developers and establish your mission

First and foremost, a Dev Acceleration team has to dig into developer pain points and the status quo without initial bias. They must know and listen to their customer: namely, developers. This is no different than any other feature team!

It’s important to come into this role with opinions that are loosely held: you can have a preconceived notion of what needs to be done, but in order to make progress, it comes down to establishing a mission and establishing relationships with engineers who can guide the team in the direction of that mission.

How that is achieved at one company can look extremely different at another!

I’ve worked with companies that had many different teams managing their own infrastructure and operated their own microservices with Kubernetes. There was a high demand for a platform engineering team that could provide services for teams to manage their own services independently. Jobber, on the other hand, has a centralized infrastructure team hosting a Rails monolith that most of our engineering teams work on simultaneously. Clearly, what engineers desired in my past experience may not be something that Jobber engineers want, need, or even think about!

When I started at Jobber, my manager had compiled a 12-page document exploring pain points within engineering. This was a great starting point to focus my energy and talk to people within engineering about the most pressing issues. Over the course of a few weeks, I chatted with many engineering managers and principal engineers to confirm the pain points in that document, and from there generated a developer survey to get feedback from the rest of the engineers in the company about their experience.

From the survey responses we received, I felt it was prudent to follow up with developers who showed some passion about the bottlenecks at the company. This usually comes from senior engineers or tech leads, but it’s important to note junior engineers' opinions as well. Seniors may be skilled enough to work around some of the challenges, but they can overlook some of them. Juniors might struggle with these hidden problems. By engaging with engineers of all kinds and encouraging them to follow through on these problems, we foster that culture that generates great quality-of-life pull requests from engineers that want to make Jobber an easier habitat to get things done.

Establishing a Developer Acceleration mission

A key question for any developer experience team to consider is: How are you actually improving the developer experience or productivity at the company? What does developer experience even mean?

Answering this question will get your team to start thinking about its mission and your metrics. For us, our mission at Jobber is: we enable engineers to ship faster. That means we’re focused on producing four types of value for developers:

Speed: we remove blockers that are slowing devs from shipping code faster
Efficiency: we reduce developer toil and build our tooling to scale
Clarity: we reduce cognitive load and make processes easier
Autonomy: we reduce dependencies and make tooling that is self-serve

These values serve as the benchmark for where we want to go. From these values, we can determine metrics that prove that we’re having an impact in our target value area.

In the future, we want to leverage frameworks like DORA or SPACE to provide quantitative and qualitative data for our work. But for the time being, we’ve kept our focus on honing our metrics to more targeted areas within our control.

It’s likely that some of the work will be a lagging indicator. You might not see results right away. Be patient: Adoption of your new, improved tooling and processes will take time.

Exploring themes and bottlenecks

With the soul-searching complete, it was time to dial in on the low-hanging fruit that could provide the most impact. We knew there were great opportunities to improve the speed at which things are done at Jobber. But we had to be mindful that we didn’t spend a lot of time doing the wrong thing to speed things up based on loose hypotheses. Personally, I think one of the worst approaches a team like this could take would be to take on a large and lengthy project filled with uncertainty and ultimately fail to deliver results - or even worse, results that no one wants!

That was almost the case with our first project, and why we live by taking small bites and working iteratively.

Improving local dev environments

Something that was addressed by many engineers in the company was difficulty getting their dev environment set up and keeping it up-to-date. Our setup can be a bit tedious, and it’s not uncommon for a tooling/language update to happen in our codebase which can break a few developers’ environments if they miss the public service announcement.

To address this, we decided to explore GitHub Codespaces and dev containers as a way to automate the dev environment and provide on-demand testing environments for engineers, product managers, and engineering managers to access. It was actually pretty easy to get Jobber running in Codespaces in a couple of days! But could it be a full-on replacement for dev environments? Would it support all of our integrations? Would it be stable enough?

We delivered a proof-of-concept, a trial run with a handful of devs, and further investigation that revealed the Codespaces experience was too tedious and required much more effort and automation to bring up to parity with a local dev environment. Ultimately, we decided that it was not worth pursuing further and would revisit in the future.

But this was a great example of pivoting quickly: we placed a small bet, got some feedback, and decided it wasn’t the best use of our time to bullet-proof a solution there. With a small team of three engineers, there were even bigger wins to be had with that involved less uncertainty.

CI/CD pipeline

The performance of the CI/CD pipeline affects every single Jobber engineer. Whether it’s a flaky test, a blocked deployment pipeline, or long-running test suites that get slower by the day, the impact of a poorly running CI/CD pipeline can silently grow and is most likely to be accepted as ‘that’s the way it is’. At Jobber, there was some degree of that but there also wasn’t a dedicated team to tackle those sorts of issues with the time needed to dig deep.

As a result, a majority of our time has been spent improving the performance of that pipeline and stabilizing that process for the future growth of more Jobber engineers to build, test, and deploy their changes.

Our biggest win so far has been reducing the pipeline time by nearly 50%. Our test times were sometimes hitting 20 mins per run for 30,000 rspec tests and 6,000 Jest tests. With some work towards more efficient usage of the testing infrastructure, we’ve brought our times down to 12 mins per run with 40,000 rspec tests and 10,000 Jest tests for roughly the same cost per run.

Prioritizing the most impactful work

We could have put all of our effort into making GitHub Codespaces our dev environment of choice, but was that guaranteed to be useful, adopted, and loved by our engineers? Tackling the low effort/high impact items may not be flashy, but when you’re a small team just starting out, it’s important to establish trust and deliver meaningful results.

What this looks like at another company will probably be different. A developer experience team will likely have a different focus based upon their developer discovery work. But it’s important that you’re not led astray by recency bias, where the last thing that bothered you seems like the priority. A Dev Acceleration team has to discern the high impact work that will affect the most people.

Where we’re headed next

As mentioned earlier, Jobber is a monolith and the majority of our teams contribute code to the same repository daily. As we scale, we want to ensure the values we espouse scales with the monolith, too. We don’t want to slow down just because we are growing.

That means maintaining speed and efficiency: we want to build our pipeline to scale for fast builds, test suites, and deploys. We average around 40-50 commits to our repository a day and we want to build a release pipeline that scales for releasing 100s of times a day.

We also want to improve autonomy and clarity. Establishing ownership over components of the codebase is a critical piece of the puzzle. We want to improve our tooling for staking ownership within the monolith and have that propagate into our observability and processes. This will make it easier to scale our day-to-day operations of the monolith.

And most importantly, we want to hear what devs have to say about their current experience. That means crafting a developer survey to hear their pain points and allow us to establish quantitative and qualitative metrics to see if we’re working on the right things, and making it easier to ship code faster.

Hiring for a Dev Acceleration team

Maybe after all this discussion, you’re starting to think about what it would be like to put together a team of your own to tackle the large, festering issues at your company. It’s tempting to just put together a Dev Acceleration team from your pool of existing developers. After all, they should know their problems best. But then you might be taking away your best developers from providing great features to the product!

I believe a mix of tenured engineers, and new, but experienced engineers is the best fit. The tenured folk have a great understanding of the day-to-day pain points of development, while the new hires come in with a fresh perspective. They offer a way of tackling problems that may be accepted as status quo, and can also distill best practices from their previous experiences.

It’s also beneficial to have developers of different types and backgrounds. Not everyone needs to be an expert in your existing tech stack.

For example, it’s completely reasonable to have some developers who are proficient in the stack paired with some site reliability engineers (SRE) or DevOps engineers. Together they can cover a large breadth of the work that comes with a Dev Acceleration team. Similar to having a mix of tenured and new developers, a blend of expertise offers the benefit of having many different perspectives for solving any challenge that might come up.

Dev Acceleration is an iterative process

Dev Acceleration teams are forging a fairly new path.

This practice has been established at quite a few larger companies. It’s now starting to make its way into growing startups that are looking to scale up their engineering department.

There are not a lot of industry standards around how developer experience teams operate and perform. It’s a lot like when DevOps was first becoming a thing.

We know we’re going to make mistakes, but we do this by making small bets and getting feedback quickly. Dev Acceleration teams really benefit from feedback loops. Software is built upon iteration, and Dev Acceleration is no exception. It will take investigation, experimentation, and being flexible to figure out what works.

Dev Acceleration teams will have the main benefit of working so closely with their customers. It’s a rewarding experience and a privilege to see and interact with the devs who are benefitting from your work on a day-to-day basis.

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Building an App in Jobber Platform

Christian Ellinger — Thu, 14 Dec 2023 17:33:07 +0000

Building the application

We are back with the second post of this three-part series. In the first post we talked about Jobber’s objectives for rebuilding the QBO integration, the principles behind it and which architectural decisions the team had to make while building the integration. The Jobber platform through the Developer Center was introduced and in this post we will dig deeper into the details which entail building the application using Jobber’s Developer Center.

Getting started

To get started with building applications in Jobber’s platform, there are a few pre-requirements:

Create a developer account in Jobber;
Create a Developer Center account. (It is important to note that the Jobber developer account is a completely separate account from the Developer Center account);

Creating the app

After creating the developer’s accounts, head over to your apps page and click on the 'NEW' button to create your first app.

When creating your application, you will be prompted for:

App name (required)
Developer name (required)
OAuth Callback URL (optional)
Manage App URL (optional)
App description (required)
Features & benefits (optional)
Scopes (required)
App logo (optional)
Gallery images (optional)
Webhooks (optional)

Any details about this app such as App Name or Description can be modified later, but it is important to select the correct scopes when starting as these currently cannot be edited later (a new app will need to be created instead). App name, Developer name, App description, Features & benefits, App logo, and Gallery images are all important pieces that'll make up the content of your app listing in Jobber's App Marketplace.

The App Name, Developer name, App Description, Manage app URL, and Features or benefits will be used by your app's listing on the marketplace (if you choose to publish your app). Check the Publishing Your Application section for more details.

The OAuth Callback URL is the URL that a Jobber user will be redirected to immediately after connecting and authorizing access to your app.

The Manage App URL is an alternative URL that can be used for any accounts that have already connected to your app but will have a need to navigate back to it to manage or configure functionalities.

Scopes are where you can set exactly what kinds of data your app will be able to read or write from Jobber accounts using the GraphQL API. They also determine exactly what is shown on the OAuth screen when a Jobber user is connecting to your app. While an app is in a Draft state, the scopes can be freely edited. However, if an app is published and would like to add more scopes, all accounts that have connected to the app previously will need to re-authorize the app. More details on editing an app can be found here.

Webhooks allow for real-time data to be sent to your application when an event occurs on models within Jobber. Webhooks are configured at the application level and will trigger when an account with an installation performs the topic. As an example, if you've configured a webhook to listen to CLIENT_CREATE, whenever a user creates a new client, the URL provided will be sent a POST request with the details in the body.

For more details on how Jobber apps work and the processes around implementing, testing, and submitting your app for review, see the guides in the Build With Jobber section. There is also an App Template Project section if you would like to download and set up a working template app to start with. It is a good way to quickly start the development using some of the pre-setup projects with UI components, authentication and API examples. In the next section, we will explore more about how the Authentication and GraphQL API works.

Authenticating the app

Jobber uses OAuth 2.0 for app authorization. OAuth 2.0 is widely adopted across various platforms, including web, desktop, and mobile applications. It's used by major applications like Facebook and GitHub.

Upon adding your app, Jobber provides a client identifier and secret. The client ID is public, and used for identification, while the secret is confidential for authenticating with the Jobber API.

To authorize your app, an admin initiates the process via Jobber's App Marketplace or a provided link:



https://api.getjobber.com/api/oauth/authorize?client_id=<CLIENT_ID>&redirect_uri=<REDIRECT_URL>&state=<STATE>

Components:

<CLIENT_ID>: Your app's Client ID from the Developer Center
<CALLBACK_URL>: The URL where Jobber redirects after authorization.
<STATE>: A random string to prevent cross-site request forgery attacks

When a user interacts with a link or clicks the Connect button in Jobber's App Marketplace, they will be asked to grant authorization to the app by clicking an "Allow Access" button. If the user is not already logged into their Jobber account, they will need to log in before reaching the authorization screen.

After approval, Jobber redirects the user to your app's callback URL with an authorization code:



https://yourapplication.com/callback?code=AUTHORIZATION_CODE&state=STATE

In case the user denies the access request, Jobber will redirect the user-agent to your application's redirect URI without any additional parameters.

Your app then obtains an access token by making a POST request to Jobber's token endpoint. The token is used for API interactions.



https://api.getjobber.com/api/oauth/token?client_id=<CLIENT_ID>&client_secret=<CLIENT_SECRET>&grant_type=authorization_code&code=<AUTHORIZATION_CODE>

If the access token expires, use the refresh token to obtain a new one. Refresh tokens may expire due to app disconnection, manual client secret rolling, re-authorization after scope change, or Refresh Token Rotation.

If both the access and refresh tokens expire, the user will need to initiate the OAuth flow again. Your app should prompt them to do so by redirecting them to Jobber and back to the app's Callback URL.

Visit our Developer Center for detailed information on how the authorization works.

Using the GraphQL Playground

Before fully building out your app you can view the GraphQL Docs and Schema and make API requests using the GraphQL Playground tool and allowing access to your tester Jobber account. To do this, click on the three dots next to your app in the Developer Center and then click Test in Playground.

This will take you through the same Oauth2 flow that a normal admin user would do, and afterwards you'll be redirected to GraphQL Playground where you can view the available queries and mutations with the DOCS button on the right, or you can download the entire Schema. Note that the scopes on your application will be respected and you may not be able to query for all of the data seen within our schema.

An example query will already be provided for you, but a second example query you could use to view the id, jobNumber, and title of every Job in your tester account is:



query SampleQuery {
  jobs {
    nodes {
      id
      jobNumber
      title
    }
  }
}

Alternatively, you can make API requests by posting your GraphQL query or mutation to https://api.getjobber.com/api/graphql. You will need to include the Access token from step 4 under the Authorization header, preceded by the word bearer.

Other important things

App-configured custom fields

Custom fields in Jobber are a popular feature, allowing various data formats to be linked to six Jobber objects: Clients, Properties, Quotes, Jobs, Invoices, and soon Team members. These fields are set up at the account level and store values at the object level. Admin users typically configure these fields, but the API also permits apps to create new custom fields with proper permissions.

Custom field configurations include settings like custom field name, automatic data transfer to related objects, default values, and data types (True/False, Numeric, Area, Dropdown, Text, Link), along with units of measure for numeric and area fields.

When an app configures a custom field, its name and logo are displayed to Jobber users whenever they encounter the field in Jobber's user interface. This enhances the user experience and association with the app. For example, if an app called “XYZ App” configured two custom fields on all job objects, the resulting user experience would look like the screenshot below.

API Versioning

API versioning is a method that Jobber uses to continuously improve its platform and features while ensuring a predictable process for third-party developers to upgrade their applications. Here's a summary of the approach:

Dangerous and breaking changes are introduced in versioned releases.
Versions are labeled with date formats (YYYY-MM-DD) and irregularly published to the changelog.
New versions are added whenever there are breaking or dangerous changes to the API.
Old versions are supported for 12 months after the release of a newer version.
If you're using an unsupported version, you'll be automatically upgraded to the earliest supported version, even if it includes breaking changes.

API rate limits

Jobber's GraphQL API has rate limits in place to ensure the stability of the platform for all interacting apps. There are two rate limiters in use:

DDoS Protection Middleware: This limits clients to 2500 requests per 5 minutes on a per app/account basis, rather than by IP address. If an app exceeds this limit, subsequent requests to the same Jobber account will receive a "429 Too Many Requests" error. This limit is generally less restrictive than the GraphQL Query Cost limit.
GraphQL Query Cost Calculation: Each app/account combination has a maximum number of points available for making queries. The cost of each query is deducted from this point pool. Points are restored over time using the leaky bucket algorithm. If queries are reasonably sized, staying within these limits is straightforward. If you find it challenging, consider introducing delays between queries and caching common results. Error handling can also help regulate queries when throttled.

To avoid rate limits, it's recommended to use pagination to collect data in batches, utilize cursor-based pagination for Relay-based queries, and avoid deeply nested queries whenever possible. If nested queries are needed, always apply pagination to connection types. Using these strategies will help you stay within the rate limits and ensure the smooth functioning of your app with Jobber's API.

Handling API Errors

Handling API error scenarios is crucial for maintaining a robust and reliable interaction with Jobber's GraphQL API. Some of the errors you might encounter are Type Nullability, Scopes & Permissions, App-level Authorization, Jobber Preventative Errors, Scalar Coercion, Rate-Limiting Errors and Inactive User Error.
In the Developer Center documentation you can find an extended explanation of these errors and how to avoid them.

Testing the application

During the application building process a Jobber Tester Account was created, linked to your app through the App Authorization flow and GraphQL queries/mutations tested in GraphiQL. Now is the time to test the application integrated with Jobber and prepare it for publication on the Jobber App Marketplace.

For your app to be published in Jobber's App Marketplace, there are typically 3 main steps to the testing process:

Internal Testing by Developer:
Thoroughly test your application internally before submitting it for review. Avoid involving existing Jobber customers in testing without coordinating with your Jobber developer representative and keep information about your app private until it's approved through Jobber's App Review process.
App Review Testing by Jobber:
After your internal testing, submit your app for review by Jobber who will verify your app's functionality and may request changes. Your App Marketplace listing details will also be reviewed and Jobber will perform various test cases, including checking proper functionality, data formatting, and data syncing with Jobber.
Beta Testing with Actual Jobber Customers:
Once approved by Jobber's App Review team, there's typically a 2-week beta testing period with selected Jobber users. Be prepared to make further changes based on feedback from these users.

By following these steps, you can ensure that your app is thoroughly tested, meets Jobber's requirements, and is ready for publication on the Jobber App Marketplace.

Load testing the QBO sync app

Load testing is an important component in our application’s release cycle. It ensures that any new feature or application we’re releasing can handle the expected load rates without performance degradation or downtime. For our QBO Sync app, we needed to ensure it could handle anticipated growth in connected users over the next few years, in addition to the expected release volume.

We split our load testing into two phases: Lightweight and Robust. The Lightweight phase aimed to quickly validate our app's ability to handle the load we expected to see, while our Robust phase would implement a tooling based approach that would provide us with more data over time. After investigating several tools we decided on building custom tooling for the lightweight phase.

Why would you build out custom tooling? We determined that building our own tool would mean writing less code, testing more of our system, and allowing us to gain more confidence in our approach.

We built a system that would trigger events inside of the main Jobber Online platform, which would then send webhooks to our app. By triggering events on test accounts inside of our main platform rather than mocking them, we were able to test a more complete picture of the application. This also meant that we didn't have to mock out our webhook authenticity check which had posed a challenge in our investigation.

Having a purpose-built solution quickly available made the most sense for the needs of this project and helped us bridge the gap of evaluating and implementing a robust load testing tool.

End-to-End testing the QBO sync app

The nature of our application being a passthrough for data between Jobber and Quickboooks Online meant we needed to come up with a novel solution for end-to-end testing. In the early stages of the project, our end to end testing was done manually by engineers following lengthy Test Case documents. One of the main goals of our end-to-end testing was to replace this manual testing with an automated, repeatable system. This would be both easier and faster to perform than manual testing, meaning that we could perform the tests more often and eventually find bugs earlier.

Conventionally, end-to-end tests would be contained inside a single application. The trigger for a test would be some interaction in the UI, and the expected result would be a change in the interface of the same application.

The trigger for most actions in our application are webhooks sent by Jobber, which are processed by our application, creating or editing data inside QuickBooks Online. This presented a challenge: both the triggers for our test events, and the results we needed to verify were outside of our application, not to mention being in separate systems themselves. We realized quickly that a conventional approach to end-to-end testing wouldn’t work for us, and we’d have to build our own system.

To test our app’s workflow, we separated our testing framework into three components.

Data Seeding: We built a rake task that creates and edits data inside of Jobber, which in turn sends webhooks to our application.
Verification: Using Rspec as a test framework, we make requests to the Quickbooks API to verify the data that was created in the first step is in the correct state in QBO. These tests are triggered by a rake task and are run separately from our unit tests.
Cleanup: The QBO API doesn't allow for deletion of all of the models (i.e. inventory items) that we are testing, instead allowing developers to reset the entirety of the data in a sandbox. We then build another small rake task to wipe the test data created by the data seeding from Jobber.

Launching the application

After completing development and testing, it is time to publish the application so that Service Providers can start using it. The publishing process involves providing the listing details to appear in Jobber’s App Marketplace, having the Jobber team review the application, and setting up an entry point for users if your application has a settings page or if you want them to manage functionalities within your application.

Providing listing details

Upon publication, the listing on the marketplace will be characterized by the utilization of the App name, Developer name, App description, Manage App URL (optional for a basic listing), Features or benefits, Gallery images, and App logo fields. The finalized appearance of the app will look like the following:

Review process

When submitting your application, Jobber will prompt you to fulfill the App Pre-Submission Checklist. For additional details on the app testing procedure, refer to the Testing An App section. To initiate a review of your app, simply click on the three dots action menu adjacent to your app within the Manage Apps view. If you have no plans to publish your app on Jobber's App Marketplace, please consult the Custom Integrations section.

During the review process, Jobber may request any changes necessary to best integrate with Jobber's ecosystem. Once the review is complete, the app will be ready for users. The review process can also be canceled at any time if desired.

Adding a Manage App button

If the application includes a settings page or another page to manage functionalities, you can leverage the Manage App URL field in our Developer Center. This allows you to unveil a new button for Jobber admin users directly from the App Marketplace listing. The button will be visible exclusively to Jobber admin users who are exploring your App Marketplace listing post their Jobber account is successfully linked to your application. More information on the Developer Center.

Summary

Building and launching an application using Jobber’s platform is straightforward and the Developer Center provides you with all the information you need.

In the third and final post, we will discuss the challenges we encountered, share some best practices for building an application on Jobber's platform, delve into the lessons learned, highlight significant successes, and outline our future plans.

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

A post by Christian do Prado Silva, Diogo Gallo, and Christian Ellinger

Building an App in Jobber Platform

Diogo Gallo — Wed, 27 Sep 2023 16:49:14 +0000

This is the first post of a three-part series

At Jobber we strive to help thousands of Home Service Pros worldwide in managing their business, and part of the mission is to rethink how they can do that more efficiently. In service of this mission, we built the Jobber’s Developer Center to empower developers to integrate and create applications to do just that.

The Jobber’s Developer Center offers a modern technology stack that facilitates the creation of applications. Companies such as CompanyCam and Thumbtack have already developed applications that enhance the Service Provider experience with Jobber. Our most recent integration was with Intuit QuickBooks Online. An internal team at Jobber was tasked with envisioning and building a seamless data synchronization between Jobber and QuickBooks Online.

Although the synchronization between the systems existed in Jobber since its inception, it was not user-friendly, causing friction for Service Providers who required cohesive data in both systems. Built entirely on the Jobber’s platform, using the GraphQL API and the Jobber Design System, the new solution expanded the capabilities of the platform and established a foundation for future applications to be built upon.

In this three-part series, we will dive into the pain points of the previous QuickBooks Online (QBO) integration, the principles we followed to develop the new application, the main architectural decisions, as well as provide details and best practices to help developers successfully build and launch their applications.

Rebuilding the QBO Integration

Nearly every small business begins by using an accounting solution before incorporating a workflow solution like Jobber. Effective accounting and finance practices are essential for running a small business successfully. We want our service providers (SPs) to be able to manage their businesses in Jobber while seamlessly integrating with QuickBooks Online, knowing that their accountants won't be pulling hairs at the end of each month.

However, the previous integration between Jobber and QBO had some drawbacks that hindered users' overall experience. The initial synchronization process was challenging and posed risks, often requiring extensive support from the Jobber Success team. It was difficult to determine which platform, Jobber or QBO, was the source of certain issues. The manual synchronization process did not meet consumer-grade expectations, such as simplicity and intuitiveness, and allowed unclear errors to accumulate. Moreover, maintaining and enhancing the integration had become increasingly difficult, resulting in long wait times for new features and bug fixes.

Early on in the project, the team decided to rebuild the integration from scratch, aiming to make it simpler, more capable, and fully self-serve. Throughout this endeavor, we established guiding principles to aid our decision-making process:

Focus on simplicity, predictability, and reliability: We can easily explain to SPs what will happen or has happened when they use the integration.
Establish Jobber as the source of truth: Optimize the integration for SPs who rely on Jobber as their primary source of information for quoting, scheduling, invoicing, and receiving payments.
Develop as a first-party application: Function as a first-party service independent of Jobber, leveraging our platform infrastructure and expanding its capabilities to support similar integrations, enabling others to build comparable solutions.
Automated and near real-time synchronization: Eliminate the need for manual syncing; every change made in Jobber will be instantly captured and pushed to QBO.

The core of the integration is an ongoing, automatic sync of data from Jobber to QBO. This process is supported by a simple, self-serve user interface that simplifies configuration settings and error handling.

Below we have some highlights of this solution. For more detailed information on how the integration works, please refer to our Jobber Help Center.

Once you connect the app and have your ongoing sync enabled, every change in Jobber will be sent to QBO and you can keep track in the Sync Activity dashboard.

A detailed view of an item that has been successfully synced shows the date and time it has been synced and also links to the respective item in Jobber and QBO.

Once an item is synced with a warning or error, the detailed view shows a clear message and provides you with actionable options to fix the problem.

In Jobber, if there is any sync activity that requires your attention you'll see an alert in your top navigation. Click the alert to view the sync activity dashboard.

During the import, if client duplicates are detected you will have the option of selecting the client in Jobber that the client in QuickBooks will be mapped to.

And if you want to change any default settings, the intuitive Settings page is where you can customize when each item syncs with Jobber.

Architectural decisions

First-party application

One of the initial architectural decisions the team had to make was how to rebuild this integration. Several tentative plans were considered for rebuilding the integration within our monolith. However, there were always constraints when it came to maintaining the existing integration while simultaneously incorporating the new one into the same codebase.

Developing an application outside of our monolith would provide the project with a greenfield opportunity to address all existing issues with minimal worry about backward compatibility. Additionally, it would allow us to take advantage of the nascent Jobber platform, expanding its capabilities to support similar accounting integrations or any other features that would benefit from the platform improvements.

Tech Stack

After deciding that it would be an external application, we had the option to select a technology stack that was better suited to its needs. Certain characteristics were prioritized in making this decision:

Performance: it should be able to perform an object sync instantaneously, i.e. in milliseconds
Scalability: it should support our current users already integrating with QBO and also support Jobber’s growth
Operability: it should be easy and efficient to operate, easy to maintain and troubleshoot, with high availability and reliability due to near real-time live sync.
Maintainability: it should be easy for the development team to include new features and resolve any issues without the need for major refactors in the future.

With those characteristics in mind, we chose Ruby and Rails for the backend, powered by a GraphQL API, React library for the front end, and Sidekiq for background job processing responsible for all the syncs. This tech stack mimics the core of our Jobber application, which allowed us to quickly begin development, and minimized the learning curve for the development team. We also took the opportunity to use the most recent version of Ruby and Rails at the time (3.1 and 7 respectively), opening opportunities to explore new features such as Native Encryption and incorporating a type checking Sorbet, built by Stripe.

Data Change Capture

To establish an automatic and real-time sync between Jobber and QBO, it was necessary to capture any changes made to objects in the Jobber application and initiate the synchronization process.

Within Jobber, we already have a mechanism for capturing data changes and notifying subscribers who can react to those changes. One of these subscribers is powered by our platform, which sends webhooks to external applications that have registered for events.

As an external application, it was crucial to not miss any of these webhooks; otherwise, the synchronization process would not be triggered, and both systems would be out of sync. To ensure the processing of all webhook requests, an AWS Lambda function was configured with a function URL— an HTTPS endpoint used for subscribing to webhook events. Upon receiving an event, the Lambda function validates the authenticity of the webhook request, ensuring it originated from Jobber, and then sends the event to an AWS SQS queue, which is eventually consumed by the QBO App.

Using AWS Lambda ensures high availability for receiving webhook requests, and utilizing AWS SQS allows for the necessary throughput to process all the updates without creating a bottleneck in the application, especially during periods of high activity.

Localstack to mimic AWS environment

We decided early on that we wanted our local development environments to match the production infrastructure as closely as possible. To do this, we used LocalStack to emulate the cloud services (AWS Lambda and AWS SQS) that our application required. This allowed us to build and test the code which interacted with these services without having to provision AWS resources for each developer, saving us both time and money.

Sorbet usage

Sorbet is a type checking gem built by Stripe, which adds static typing to Ruby. We decided to adopt it for this project to improve code readability, but also as a proof of concept before potentially adopting it into the larger Jobber codebase. In practice, we found both benefits and drawbacks to using Sorbet, as is often the case with any tool. The improved code readability and method introspection were noticeably beneficial during development, however, the new system made bringing additional developers onto the project difficult as they were unfamiliar with the new type system.

Summary

As we can see in this first post of the three-part series, there are several factors and decisions involved while building an integration application. Jobber created the Developer Center with that in mind, making it straightforward for any developer to develop their applications and integrate with Jobber. In the second post we will get into the technical details of how you also could build an application and everything you need to do from the app creation to the launch.

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

A post by Christian Ellinger, Christian do Prado Silva and Diogo Gallo

Goals Hurt

John Zittlau — Wed, 09 Aug 2023 17:05:32 +0000

I was recently listening to a podcast where the speaker was someone being introduced as very successful. By many metrics they were. They had lots of money, C-level at a respected company, had competed in world cups and been nationally ranked in a number of diverse sports. Yes, they had accomplished lots. As they were sharing their thoughts on how to succeed, one of their primary thesis was to set big audacious personal goals. Don't set a goal of being worth 1 Million, set a goal of personally being worth 100 Million (or maybe even a billion). The reason for a very hard goal (they said) is that if you set an easier personal goal, then when you achieve it you'll be lost. Cast adrift. Have nothing to work for. You'll have to spend time finding a new goal and you may find that you've "wasted" your time on your old goal as it doesn't lead directly to the new one.

I could not relate to this way of thinking. I have no problem finding valuable things to do without audacious goals and by many measures I view myself as successful. Further, I do not believe that formally setting goals has contributed to my success. In fact, as you read further, you'll see that I believe formal goal setting to be more of a hindrance than a benefit for me.

We're constantly hearing from "successful" people that they owe their success in large part to setting goals. That the goals gave them the drive to persevere. To push through obstacles and make those goals a reality. I'm sure they believe that. I'm not sure it is a universal constant. Or that it is the one-true-way to success.

The belief that setting personal goals leads to success is so pervasive that it has taken on the status of a law of nature. Set big goals. Drive to achieve them. Profit. Many companies see personal goal setting as a critical part of career development and the road to producing high-performing individuals. Everyone is tasked with creating goals for themselves and then measured on how well they achieve those goals.

This is such a universal belief that it is only now, after almost three decades of professional life, that I've finally developed the confidence and courage to say that goals don't work for me. For most of my career, I've figured something must be wrong with me. Everyone says set goals and win. I set goals and could care less about them. The presence of a goal does almost nothing to push me along. And if I have been forced to set a big goal and don't achieve it, I feel demotivated. But not demotivated in a way that makes me want to try and achieve my next goal. Demotivated in a way that makes me want to simply give up.

Let's be clear on one thing here. I'm talking about personal goals. And also mostly the bigger goals (anything with a timespan larger than a month). I do set small goals, but not as a "forcing function", but rather as a prioritization tool. I also see team goals differently. Setting reasonable goals for a team is valuable. It helps ensure the team is moving in the same direction. It likely is helping team's prioritize, much as small personal goals help me.

As I've said, I do set little goals for the day or weekend or maybe week (a month on the extreme outside). That does help me prioritize what to do, but it doesn't drive me to do anything. And if I don't achieve my daily goal, I'm OK with that. As long as I'm happy with how I spent my time, then I'm good. Maybe that is the one goal that works for me. Be sure I'm happy with where I put my energies for that period of time. At work, this means I'm happy when I'm confident that what I did helped Jobber and was among the most important things I could be doing at that time.

On the weekend, being happy with how I spent my time could mean that I'm happy I spent a bunch of hours in front of the TV or video game. Or maybe I'm happy having done some woodworking project. Or I created an awesome Sunday meal. Or maybe I did spend the weekend diving into some new learning that will help my career. Clearly, I'm not a driven person. The journey is at least as important as the destination.

I remember the first time I worked somewhere that asked me to set career goals. I'd never done it before and was happy with where I was in my career, but set the goal anyway. Then a year later I hadn't achieved the goals I'd set. Boy did I not like that. But as I said above, it didn't motivate me to change anything. It just made me question if I wanted to stay at the company that was forcing me to set goals. It was already clear to me then that setting goals was not helping me achieve in any way and in fact was just making me less happy with life.

I suspect we hear about this path to success a disproportionate amount of time. It seems far less motivational to hear from those who "just fell into something". That doesn't provide for any life hacks to take. Setting goals is something anyone can do. Getting lucky isn't. So we hear from those who have a "solution" we all can do. The simple advice to riches draws attention. It promises a quick fix. Who doesn't want a quick fix that will put them on magazine covers.

I believe now that in many cases those who credit setting goals as the driver to their success succeeded not because they set goals, but that they have a "goal driven" personality. They are ready to put everything else aside to "succeed". Setting goals isn't changing their approach to life. Their approach to life includes setting goals.

There are others, like me, who are happy to let life take us where it will. Where setting goals is contradictory to what wakes us up in the morning. Where defining a goal is hard and caring about that goal is even harder. For us, when the company says "create goals" we grumble and do it because we have to, but we don't like it. And it doesn't help us to succeed. If anything, when we review the goals, we're irritated and stressed that they are there and want to find a safe place where we can grow without having decided how we will grow in advance.

I'm now a Distinguished Software Engineer. I think I can point to that as evidence that I've "succeeded" as a developer. Can I credit goal setting for getting here? Absolutely not. In fact I had to fight through hating the goal setting process to continue on as a developer. The temptation to find someplace safe without goals has always been strong.

You may have noticed that I put "succeed" in quotes everywhere. That is because in too many cases when I see someone has "succeeded", it is in one facet of their life at the expense of some other. Even I did that above where I tied my title to being a signifier of success. Often money/career is viewed as success. Family/friends don't even enter the conversation.

I confidently say I've succeeded without the quotes. I may not have Elon Musk levels of money, but I get by. I may not have huge followers on some social network, but I have solid friends. And I have a solid family. Everything in balance. I'm happy. At least until the next time I'm asked to set a personal goal.

So who's with me? Are goals essential for success? Do they help or hinder you?

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Jobber's React Native migration - Success!

Ryan Jones — Wed, 21 Jun 2023 17:44:34 +0000

January ‘23 was an exciting month. Not only did we raise a Series D but we launched our new version of the Jobber app which is now powered by React Native. This post is mainly coming from the engineering lens. If that’s your jam, continue reading!

I’m going to try and provide as much insight as I can to what happens ‘behind the curtains’ on one of these types of projects.

Don’t care about the story? Jump to the results.

The decision

Jobber has been growing fast. With that growth has come with it a set of expectations on both of our online and mobile offerings. Our mobile offering in particular needed some TLC (Tender Loving Care). We’d been running on the ember.js framework (wrapped webview) which we had launched back in Aug 2015.

The reality is that ember.js wasn’t built for mobile apps and it was starting to show. Basic native functionality that customers expect (pull down to refresh, swipe for nav) either took a long time to build or just wasn’t giving us that ‘native mobile app’ feel. As the mobile app volume started to increase we started to hit odd edge cases/errors and it was becoming really tough to debug the issues. In addition to all of that it was starting to get really hard to have multiple teams working in the app’s codebase.

The state of the app was normal for a startup that was trying to find product-market fit but we were past that stage and our customers deserved better. In October ‘21 we decided to do an ‘all stop’ and focus 80% of our effort to accelerate the rebuild of our mobile app.

History

Alright, I know what you’re thinking. ‘A rebuild!? Are you nuts?’. I agree with you. I absolutely, to the core of my being, hate re-writes. I’ve seen more than enough go sideways in my career. Everything from under estimating to scope increases to leadership losing their appetite for a hold on feature work.

But, let’s rewind a bit. Context matters. We didn’t go into this one blind. We did a lot of exploration before we pulled the pin, here’s what we did before we made the decision to accelerate the build:

2020 - March - Decision to move forward with React Native as our mobile framework
2020 - April - Explore how we can migrate to React Native without a big bang. Pulled React Native into the app ‘behind the scenes’
2020 - August - Decision to adopt GraphQL as our API layer (to power mobile, web and our public API)
2020 - September - First 2 features launched in React Native
2021 - January - A few more features launched in React Native
2021 - March - Replaced the Navigation
2021 - July - A few more features rebuilt in React Native
2021 - October - Decision to Accelerate build

We had done a lot of way finding along the way and even started to migrate the app without having to do a ‘big bang’. This wasn’t a full YOLO moment, but we definitely had some hurdles to overcome.

The scope

Alright, so we’re going to accelerate the build. What does this mean? Well, it means a lot of other things are going to need to be accelerated. Here’s the list of known hurdles that we had in front of us:

Train 50+ software engineers in React Native
Train 50+ software engineers in GraphQL
We need to design the new experience (we opted to improve a few key experiences)
Build out our mobile design system
Hire 30 more software engineers in the middle of the build and ramp them up 😱

I remember laying in bed the night after we (myself, CTO and VP, Product) made the call to accelerate the build, staring at the ceiling with a mix of excitement and dread. On one hand, this would be a huge win for the mobile app, on the other hand we had massive hurdles to overcome. Any one of the hurdles by itself would be tough to overcome.

The goals

These types of projects are tough because you’re trying to accomplish multiple things at once and you end up with competing priorities. We landed on these prioritized goals for the project:

The app is easier and more productive to develop consumer-grade experiences in
Our app is more reliable – it has fewer bugs, fewer crashes, fewer white screens (unhandled exceptions)
We have improved the user experience for our SPs
Our calendar is more powerful
Our app meets a minimum bar for performance
All of the above is live by Aug 30/2022
There is no ember.js left in the app by Feb 15/2023

We landed on Aug 30 based on a few high level estimates from a few teams who had the most experience with React Native and GraphQL. Since we had made the decision to accelerate, it was better to start executing and refine those dates as we progressed.

Every week we would meet to review each of the above goals and any associated metrics as a 🔴/🟡/🟢 projection. Some of them are tough to measure (e.g. How do you measure engineering productivity? Lack of observability in the old app ruled out many direct comparisons on performance and stability).

The build

We’re a big fan of the triad model at Jobber where we have equal discipline (eng/product/design) representation at the table as we build. We had a triad overseeing the whole project. Within the project we had multiple teams shipping within their areas (e.g. Our fintech teams would re-write the fintech functionality).

I absolutely hate ‘like for like’ conversions so we opted to add some improved functionality in this project. Specifically a few areas that we couldn’t have made better in the old ember.js framework. They were an improved schedule/calendar, revamped settings drawer, revamped file upload control and much better form UX. We accepted the scope increase for the benefit of our customers.

From November 2022 to Feb 2023 you can imagine a flurry of activity of designing, building, collaborating and shipping to the app. Here’s the noteworthy milestones.

April 2022 - Alpha ships
July 2022 - Beta ships
August 2022 - Code freeze for GA (‘Opt in to new version’ for users)
Sept 2022 - Rolled out new functionality to new users
Oct 2022 - First wave of forced rollout
Nov 2022 - Second wave of forced rollout
Nov 2022 - Oh 💩- Low perf device issues
Dec 2022 - Third wave of forced rollout
Jan 2023 - Final wave of forced rollout to low perf devices
Feb 2023 - Ember.js fully removed

Ultimately we managed to ship the new React Native functionality by August but it took a few more months to fully roll it out and remove ember.js. This was great as we had the better version of the app in our customers hands as soon as possible.

Things that went well

Roll out React Native beside ember.js

We rolled out the base framework underneath ember.js to help prove out the build and migrated some core components such as the navigation bar, home screen and communication center.

This gave us enough confidence in the tech to push forward and make the full conversion.

Tech lead syncs

Every week the tech leads from the teams would sync up and either highlight upcoming changes or raise a hand to help get support from other teams.

Strike teams on common components

As common components were identified, having an individual contributor from each of the teams work on the shared components as a short-term strike team worked well.

Mobile components team

We spun up a new team to help support the mobile iOS/Android work and added to our components team which helped accelerate all of the other teams. This approach was so valuable that we’ve mirrored a similar approach as we migrate our online interface to React.js

User testing

Testing functionality with users and non-users gave us early feedback that allowed us to iterate before we invested in the build.

Through the project, our entire processes around design review and user testing leveled up. We continue to benefit from our newly established best practices around early user engagement feedback.

Observability

Purchased and implemented sentry to provide crash/performance metrics and make finding, assigning and fixing issues easier for teams.

Team health

Engage teams with qualitative surveys to understand if our code health, build speed and release process was improving.

Migration metrics

We kept a keen eye on our technical metrics to make sure that we were progressing to plan. A few of these were:

Ember.js vs. React Native routes converted
GraphQL adoption metrics

Bitrise

Unified both iOS and Android building to bitrise for our mobile build pipeline. Much better than the older Buddy Build system which was purchased by Apple, put into hibernation, and then shut down by Apple.

Runway

Implemented runway to help with our mobile release process.

React native upgrade

We started to hit a few snags along the way due to being behind a few releases of React Native. Popping off the upgrade helped us get over those hurdles.

Split

We implemented split on mobile which helped us control the rollout and alpha/betas with feature flags.

Things that didn’t go well

Every project of this size will have bumps and bruises along the way.

Fine grained iterative deploys

We originally wanted to deploy each screen behind feature flags to allow the most granular level of deploys, but when you mixed in different SaaS plan types it became really hard to manage. We also ran into complex navigation issues between ember.js nav stack and React Native’s nav stack. We did use feature flags but we ended up leaving them at the SaaS plan level.

We took 3 runs at this before bailing. We just couldn’t manage the complexity of multiple frameworks effectively.

1 dedicated principal IC

We should have assigned 1 principal engineer to the project to help coordinate and overcome some of the engineering challenges. We had 3 key individual contributors take on this mantle 1/3rd of the way through the project, but it would have been better to assign this early on.

Oh 💩- Low performance device issues

Woof. We took a few app review hits on this one. We started to see a few bad reviews come in regarding performance and we realized that we didn’t have a wide enough set of test devices on Android. We didn’t see this in our alpha/beta group as we learned that they didn’t reflect the device spread for the general release. The other issue was that our simulators didn’t reflect the realities of the slower devices like we had assumed.

We rectified this quickly by:

Implementing a bunch of optimizations via memoization/use callback (using Shopify’s lovely perf libs ❤️🇨🇦) to improve performance.
Purchasing and distributing the lowest performing Android phones (the slowest android phones on the market) to every team to be used for all testing going forward.
Build out a test framework on AWS’s Mobile device farm w/ Appium to smoke test across many devices.

Datadog migration

We needed to migrate from New Relic to Datadog through Oct/Dec due to New Relic not willing to budge on user licenses. This was expected but really hampered a few of our teams during the project.

Features slipped in

We had 1-2 teams that opted to hold off on converting their section of the app to pop off a few things they had on their plate. This squished them at the end of the project. We ended up adding 3-4 more experienced React Native engineers to the teams to hit the deadline.

The results

The data

It’s been a few months since we’ve launched and now it’s time to present the outcomes! Let’s cycle back to the goals:

The app is easier and more productive to develop consumer-grade experiences in. We fast followed with a slew of new features such as our new map view, offline mode, arrival windows and more! ✅
Our qualitative data from the teams saw our health of codebase, speed (team), and ease of release hit some all time highs. We saw a slight dip in the health of codebase category post-launch due to moving into different codebases unrelated to this project.
Our app is more reliable – it has fewer bugs, fewer crashes, fewer white screens (unhandled exceptions) ✅
Holding well above 99.9% crash free sessions - a lack of observability between the previous app confounds direct comparison, but we are confident we were well below this threshold.
Since launch, this has further improved from ~99.95 to ~99.98.
We have improved the user experience for our SPs ✅
App store
All of our app store ratings are at, and remain at, an all time high
- Android (Google Play) 🇺🇸 4.4 -> 4.8
- Android (Google Play) 🇨🇦 4.1 -> 4.3
- iOS (App Store) 🇺🇸 4.6 -> 4.7
- iOS (App Store) 🇨🇦 4.5 -> 4.6
AARRR metrics
- Approved across the board (very large improvements in activation and conversion metrics)
Our calendar is more powerful ✅.

Old calendar screen

New calendar screen

Our app meets a minimum bar for performance ✅
Cold starts - We’re coming for you android. We’re working to grind that down even more. Median start times from ~15s to ~5s.
All of the above is live by Aug 30/2022 ✅
There is no ember.js left in the app by Feb 15/2022 ✅

The beauty

Before I close here’s a few more comparisons 😍. The new style and UX has received rave reviews from our customers.

Quote Screen

Old quote screen

New quote screen

Client Screen

Old client screen

New client screen

Visit Screen

Old visit screen

New visit screen

Other interesting metrics

GraphQL adoption

GraphQL API usage/surface area increased dramatically. The data stopped when we migrated over to Datadog in Nov but we’re at almost ~80% GraphQL traffic now with our goal to deprecate the REST API by EOY. Check out our public GraphQL API!.

Our GraphQL API covers almost all of the functionality available in Jobber’s online and mobile offering.

Mobile build time

Our mobile build times were cut in half 😱

And, we went from deploying our app every few weeks to shipping a new version every week.

Closing

I’m incredibly proud of the team and what we’ve delivered. Our customers love the new mobile app and we’ve already shipped a ton of new features in record time! Check out our changelog (Mobile App).

We have a lot of future plans for Jobber’s mobile and online offering. Enjoy React? React Native? GraphQL? Rails? Helping small businesses? Come join us at Jobber! We just raised a Series D to grow the business and become the #1 market leader in the home services space.

Thanks for reading! Want to chat? Drop a comment or shoot me an email (available in my profile).

How We Sped Up Rubocop Linting in our CI by 22x

Jc — Fri, 12 May 2023 18:23:53 +0000

At Jobber, we have been utilizing the GitHub merge queue as a way to run additional checks on code that is about to be merged - and we want this merge queue step to be fast (the target is under five minutes).

We realized it would be very useful to have our Rubocop linting run in the merge queue, particularly when there were rule changes or new custom rules added. The problem is that the linting step takes nearly 7 minutes to run on our largest codebase- much too long for our merge queue target.

Investigating Caching

The way Rubocop was being invoked in CI was with the following command:

bundle exec rubocop

But what about caching? Without explicit management of data from previous jobs, Rubocop would be starting from scratch on every CI run. Does it support caching, and could we leverage that?

It turns out that Rubocop actually has a solid caching implementation that takes care of all the heavy lifting, including cache invalidation:

Later runs will be able to retrieve this information and present the stored information instead of inspecting the file again. This will be done if the cache for the file is still valid, which it is if there are no changes in:

the contents of the inspected file

RuboCop configuration for the file

the options given to rubocop, with some exceptions that have no bearing on which offenses are reported

the Ruby version used to invoke rubocop

version of the rubocop program (or to be precise, anything in the source code of the invoked rubocop program)

The cache is automatically pruned based on file count:

Each time a file has changed, its offenses will be stored under a new key in the cache. This means that the cache will continue to grow until we do something to stop it. The configuration parameter AllCops: MaxFilesInCache sets a limit, and when the number of files in the cache exceeds that limit, the oldest files will be automatically removed from the cache.

This is amazing - a well thought-out cache invalidation strategy! The second point related to file changes getting stored under a new key doesn’t really help us though - the CI cache mechanism is immutable.

Leveraging Rubocop Caching in CI

We can’t directly ask Rubocop what it’s going to do ahead of time (there’s no API for its caching behavior), so how do we deterministically generate a cache key for our immutable cross-workflow cache that changes in lock-step with Rubocop’s cache invalidation logic?

Periodic Invalidation

Can we side-step that problem and just re-generate the cache periodically? Maybe daily, or weekly, and re-use it across all CI runs? Sure! That would certainly help - but it has the following limitations:

Cache hits decrease over time as files are modified. Probably not a problem unless a large swathe of the codebase is modified within the cache period (something like a linting autofix, or a refactor / rename).
If Rubocop decides to invalidate the cache, you’ll be right back to full-length linting durations until the next cache period occurs. The most common trigger for this is a change to Rubocop configuration.
The first run after each cache period will be full-length.
Shortening the cache period to mitigate some of the above issues has the side effect of increasing the amount of cache storage consumed by your project.

Intelligent Dynamic Invalidation

What if we could integrate Rubocop’s internal cache invalidation logic with the CI’s cache invalidation logic? The limitations turn into a single bullet point:

Cache hits decrease over time as files are modified. Probably not a problem unless a large swathe of the codebase is modified within the cache period (something like a linting autofix, or a refactor / rename).

Note that the CI service will typically expire a cache after a maximum number of days. In our case this happens every 15 days, and so there is a natural “reset” that catches the slow cache hit decline over time as files are modified.

Here's how Jobber is powering our CI cache invalidation with Rubocop’s logic!

Before you restore the rubocop cache directory (~/.cache/rubocop_cache), lint a single dedicated file using the exact same command and configuration that the full linting step uses.
Inspect what Rubocop wrote into the cache directory, and generate your cache key as a hash of that information - at this point, proceed with the normal restore, run, persist pattern.

Here’s how you get the text you want to hash - assuming you used a file that is highly unlikely to change for your detection, this essentially represents a Rubocop cache key:

$ find ~/.cache/rubocop_cache -type f
/home/circleci/.cache/rubocop_cache/c21eac4b5c1ceb0445943396a341eadb756f46cf/7a1221dfb74d1bb683162bcc22951148cd32f1c9

Output that to a file (rubocop_cache_key) and hash it, combine it with other environment keys, and you get a robust cache key!

Example cache key:

rubocop-v1-{{ arch }}-ruby_<< pipeline.parameters.ruby_version >>-{{ checksum "rubocop_cache_key" }}

Cache Key Part	Description
`rubocop`	The descriptor of the cache key - this one is intended to be unique for rubocop purposes.
`v1`	A manual version number - bump this up when there’s unexpected issues and you want a straight-forward way to explicitly invalidate the cache.
`{{ arch }}`	CircleCI notation for the architecture, such as `arch1-linux-amd64-6_85`.
`ruby_<< pipeline.parameters.ruby_version >>`	The ruby version - don’t try and share caches across ruby versions. Rubocop would almost certainly invalidate the cache in this case as well, but in our case, our setup workflow detects the Ruby version and passes it onwards as a pipeline parameter so we might as well bake it in.
`{{ checksum "rubocop_cache_key" }}`	This is both the “intelligent” and the “dynamic” part - it builds on the intelligent Rubocop invalidation logic, and is dynamic because this isn’t hashing text directly under source control. See the examples below for how to generate the `rubocop_cache_keyfile`.

Putting It All Together

So now we have a suitable cache key - what does it look like used in a CircleCI workflow (the following is a partial example of a CircleCI configuration file)?

references:
  detect_rubocop_cache_key: &detect_rubocop_cache_key
    run:
      name: Detect rubocop cache key
      command: bundle exec rubocop example.rb >/dev/null 2>&1 && find ~/.cache/rubocop_cache -type f > rubocop_cache_key && cat rubocop_cache_key

  restore_rubocop_cache: &restore_rubocop_cache
    restore_cache:
      name: Restore rubocop cache
      keys:
        - &rubocop_cache_key rubocop-v1-{{ arch }}-ruby_<< pipeline.parameters.ruby_version >>-{{ checksum "rubocop_cache_key" }}

jobs:
  lint_rubocop:
    - *bundle_install
    - *detect_rubocop_cache_key
    - *restore_rubocop_cache
    - run
        name: Rubocop linting
        command: bundle exec rubocop
    - save_cache:
        name: Save rubocop cache
        key: *rubocop_cache_key
        paths:
          - ~/.cache/rubocop_cache

Note for Very Large Projects

If your file count is close to 20k, you’ll want to tune MaxFilesInCache to be your max file count plus a percentage to accommodate cache misses (files changing over time, between cache invalidations).

Further Improvement Potential

Once you’ve optimized the amount of work your CI is doing for linting, you can get further gains through parallelization of that work - either the multi-threading kind, or the horizontal scaling kind (both involve the same amount of work, but leveraging more hardware to complete that work faster - usually at a monetary cost).

Performance Improvement Results

Before caching, linting took 476 seconds.

After caching, linting takes 22 seconds.

The result (476 / 22 = 21.6): 22x faster - easily fast enough to run a full linting check in our merge queue!

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Custom Google Map Markers on iOS with React Native Maps

Michael Davies — Thu, 26 Jan 2023 16:28:40 +0000

spoiler alert: It is possible! And if you are using react-native-reanimated in your app, it may be the culprit...

1. Preface
2. The Error in Question
3. The Broken Implementation
4. Discovery
5. Solution #1
6. Solution #2
7. Solution Trade Offs
8. TL;DR
9. About Jobber
10. Footnotes and References

NOTE: If you don't want to read this entire blog post, jump to the TL;DR.

1. Preface

So, your team is creating some form of mapping software in React Native, and you've decided to leverage react-native-maps (RNM) to seamlessly implement Google Maps (GM) on iOS and Android applications. As with any mapping software worth its salt, you want to have custom map markers - with RNM that's easy, right? Unfortunately, you might find this isn't the case.

After creating a custom React Native (RN) component to use as a child of RNM's <Marker /> component, you add it to your map view as per the RNM documentation, and you boot your iOS simulator to see your custom pins. Alas, there is a chance you are going to encounter this error:

Exception thrown while executing UI block: * -[__NSDictionaryM setObject:forKeyedSubscript:]: key cannot be nil

This is the scenario that my team and I found ourselves in. In this blog post I will outline the two solutions we developed that enabled us to use custom pins with GM on iOS in RN, using RNM. It is worth noting that I haven't ascertained the true root cause of this issue, however, I will update this blog if that changes.

2. The Error in Question

Exception thrown while executing UI block...

Like many other software errors it's not immediately clear what the problem is. Surprisingly, a Google search of the error doesn't return many results (~94 at the time of writing).

There exist a handful of unanswered Stack Overflow questions and a few issues that have been posted to the RNM GitHub repository, almost all of which invariably reference the largest thread on this error I have found.

Reading the limited resources on this issue, you might see answers such as:

The <Marker /> component can't have children.

You need to use Apple Maps as the provider for iOS and Google Maps as the provider for Android.

The native implementation of Google Map's SDK and Apple's MapKit are incompatible, so using custom markers will cause React Native to break.

Or my personal favourite, courtesy of ChatGPT:

While using provider={PROVIDER_GOOGLE}, you can not use custom pins on iOS with react-native-maps.

To generalize, it seemed as though the consensus was that you couldn't use custom pins with GM on iOS. The most commonly recommended and accepted solution is using the respective map providers for both platforms. This was something our team wanted to avoid, as to provide a consistent experience to our customers across platforms.

Through failure and exploration our team found two solutions for this error¹, and I hope that by writing this I am able to contribute to the discussion and help others overcome this error² in their own projects.

3. The Broken Implementation

To provide context, our code looked something like this when we encountered the error³.

Note: To make this easy to copy and paste for testing, I have attempted to make these code snippets as generic as possible, with limited dependencies.

Our custom marker component

interface CustomMarkerProps {
  label: string;
}

// Using text as a marker for simplicity. It has the same behaviour (with respect to the error) as an SVG or Icon component.
export function CustomMarker(props: CustomMarkerProps): JSX.Element {
  return (
    <View>
      <Text>{props.label}</Text>
    </View>
  );
}

Our component that uses React Native Reanimated (RNR)

For context, this component will render over the map (absolutely positioned) telling the user that there are no markers on the map. It is built with RNR. It fades in from the top of the screen, ultimately coming to rest 50 pixels below the top of the screen.

export function ReanimatedBubble(): JSX.Element {

  const animatedStyle = useAnimatedStyle(() => {
    return {
      top: withTiming(50, {
        duration: 100,
        easing: Easing.bezier(...EASE_CUBIC_IN_OUT),
      }),
    };
  }, []);


  return (
    <Animated.View entering={FadeInUp} exiting={FadeInOut} style={animatedStyle}>
      <View>
        <Text>{"There are no markers on the map"}</Text>
      </View>
    </Animated.View>
  )
}

Our map view component

const styles = StyleSheet.create({
  container: {
    ...StyleSheet.absoluteFillObject,
    height: "100%",
    width: "100%",
    justifyContent: 'flex-end',
    alignItems: 'center',
  },
  map: {
    ...StyleSheet.absoluteFillObject,
  },
});

type MarkerType {
  id: number;
  coords: {
    latitude: number;
    longitude: number;
  };
}

export function Map(): JSX.Element {
  const [markers, setMarkers] = useState(generateRandomMarkers());

  function generateNewMarkers() {
    setMarkers(generateRandomMarkers())
  }

  return (
    <View style={styles.container}>
      <MapView
        style={styles.map}
        provider={PROVIDER_GOOGLE}
        region={{
          latitude: 55.05,
          longitude: -90.32,
          latitudeDelta: 12.5,
          longitudeDelta: 35,
        }}>
      >
        {markers.map((marker, index) => (
          <Marker key={index} coordinate={marker.coords}>
            <CustomMarker label={marker.id} />
          </Marker>
        ))}
      </MapView>
      {markers.length === 0 && (
        <ReanimatedBubble />
      )}
      <Button
        title="PRESS ME TO CHANGE MARKERS"
        onPress{() => {
          generateNewMarkers();
        }}
      />
    </View>
  )
}

// returns an array with [0, 100] markers
function generateRandomMarkers(): MarkerType[] {
  const randInt = Math.floor(Math.random() * 101);

  const markers = [];
  for (let i=0; i < randInt; i++) {
    const marker = {
      id: i,
      coords: generateRandomCoords(),
    }
  }
}

// returns random coordinates within Canada
function generateRandomCoords() {
  const MIN_LATITUDE = 42.33173;
  const MAX_LATITUDE = 67.770579;
  const MIN_LONGITUDE = -125.004;
  const MAX_LONGITUDE = -55.6362603;

  const latitude = MIN_LATITUDE + (MAX_LATITUDE - MIN_LATITUDE) * Math.random();
  const longitude =
    MIN_LONGITUDE + (MAX_LONGITUDE - MIN_LONGITUDE) * Math.random();

  const coords = {
    latitude,
    longitude,
  };

  return coords;
}

Using the code snippets above, on initial render of the map, we see the following on an iOS simulator:

So far, so good! We have custom markers. Now, having the markers change is a critical piece of functionality.

I tap the "PRESS ME TO CHANGE MARKERS BUTTON" to see new data being visualized on the map...

The application crashes, we have hit the error.

4. Discovery: how we approached the problem

After encountering, then researching this error, one comment stood out to us.

You cannot render custom markers on iOS while using provider={PROVIDER_GOOGLE}.

With the broken implementation we had, we recognized this as a rather glaring contradiction. We did render custom markers on the map's initial render, therefore it's possible. So what's going wrong?

In some of the online threads, there is mention of developers running into this error only after they had started using RNR in their preexisting map applications, or after version changes of RNR.

Furthermore, the error states key cannot be nil. So there is somewhere in our implementation where we were dropping keys - in our case, the indices in the marker array we iterate over in the map view.

These two ideas led us to two possible solutions we wanted to test:

Is RNR breaking our map somehow?
Can we manage the marker array in such a way that we don't drop indices, thus avoiding the error?

5. Solution #1: RNR is breaking the map, let's fix it

From what we had seen online, we had a hunch that RNR may be breaking our app.

To confirm this, we created an empty RN project to start from scratch. We created a proof of concept map to run on iOS that was essentially the same as what you see in the above code snippets, but without RNR or any animated components. RNM was the only library we added to the project, and the app was able to render custom pins that changed in a random fashion - confirming that it is, in fact, possible.

To demonstrate it was RNR creating the conflict, we added the animated components into the app with the smallest incremental changes, rebuilding the iOS simulator after each step.

If you'd like to try this, you can try the following:

Install RNR
Create the most rudimentary animated component (i.e. text wrapped in an animated view), but don't render it with your map view.
Add your animated component to your map.
Piece by piece, use more of the API's methods for a more complex animated component

In doing this, we were surprised to learn that it was only specific methods within RNR that threw the error. In our case specifically, two props were to blame:

1. entering={FadeInUp}
2. exiting={FadeInOut}

To remove those props while preserving the animated functionality of the "no markers" component and solve the error, we refactored as follows:

export function ReanimatedBubble(): JSX.Element {
  const heightPadding = 50;
  const top = useSharedValue(0);
  const opacity = useSharedValue(0);

  const animatedStyle = useAnimatedStyle(() => {
    return {
      top: withTiming(top.value, {
        duration: 100,
        easing: Easing.bezier(...EASE_CUBIC_IN_OUT),
      }),
      opacity: withTiming(opacity.value, {
        duration: 100,
        easing: Easing.bezier(...EASE_CUBIC_IN_OUT),
      }),
    };
  }, []);

  useEffect(() => {
    top.value = heightPadding;
    opacity.value = 1;
  }, []);


  return (
    <Animated.View style={animatedStyle}>
      <View>
        <Text>{"There are no markers on the map"}</Text>
      </View>
    </Animated.View>
  )
}

On the iOS simulator with these changes, the map now works as expected!

6. Solution #2: buffer the marker array

Given the error is complaining about keys being nil, we investigated what key(s) were becoming nil and when. Looking at our custom marker component, we're using the indices of the marker objects within the array as the key prop in the <Marker /> component.

In testing how changes to the marker object array affects the keys being used to render the markers on the map, we discovered the following:

You can render an array of n objects as custom markers on the initial load of the map

This makes sense, we knew this from the beginning!

Changing the array such that n' >= n does not crash the app.

Again, given the error, this makes sense. In this case we aren't dropping any keys previously used for the markers. Adding additional markers isn't an issue.

Changing the array such that n' < n crashes the app

Ah ha! This confirms that it is in fact the array indices used as keys for the markers that are crashing the app when they don't exist anymore.

In other words, custom markers can change when the marker data changes, but only when the new set of markers is equal to or greater in length than the previously rendered marker array. If the new array is shorter, you will hit the error because marker indices that once existed become undefined, therefore some keys are nil.

To confirm it was the marker array indices used as keys becoming undefined we used two different methods.

We used the timeless debugging method of console logging. When mapping over the marker array in the <MapView /> we logged the indices being used. Sure enough, when the marker array decreased in length we saw undefined being console logged for missing marker objects, and the app crashed.
As a more methodical approach - we used Flipper. Adding breakpoints within the map(() => {}) sequence, we went through the iteration step by step, looking at what data was being used. Again, in the instance of marker objects being removed from the array, we saw undefined keys and the app crashed.

To fix this, we managed the array of marker objects such that it had a constant size.

Firstly, we made a hook that looks something like this:

const BUFFERED_MARKER_ARRAY_SIZE = 100;

const BUFFER_MARKER = {
  id: -1,
  coords: {
    latitude: -91, // this is not in the domain of latitudes, i.e. it won't appear on the map
    longitude: -181, // this is not in the domain of longitudes, i.e. it won't appear on the map
  },
}

const bufferedMarkerArray = Array.from(
  { length: MARKER_ARRAY_BUFFER_SIZE },
  () => PLACE_HOLDER_MARKER,
);

export function useBufferedMarkers(markers: MarkerType[]): MarkerType[] {
  const markerRef = useRef(bufferedMarkerArray);

  useEffect(() => {
    markers.forEach((marker, index) => {
      markerRef.current[index] = marker;
    });
  }, [markers]);

  return markerRef.current
}

Then, we use this hook in our <MapView /> component:

const styles = StyleSheet.create({
  container: {
    ...StyleSheet.absoluteFillObject,
    height: "100%",
    width: "100%",
    justifyContent: 'flex-end',
    alignItems: 'center',
  },
  map: {
    ...StyleSheet.absoluteFillObject,
  },
});

type MarkerType {
  id: number;
  coords: {
    latitude: number;
    longitude: number;
  };
}

export function Map(): JSX.Element {
  const [markers, setMarkers] = useState(generateRandomMarkers());

  function generateNewMarkers() {
    setMarkers(generateRandomMarkers())
  }

  const bufferedMarkers = useBufferedMarkers(markers);

  return (
    <View style={styles.container}>
      <MapView
        style={styles.map}
        provider={PROVIDER_GOOGLE}
        region={{
          latitude: 55.05,
          longitude: -90.32,
          latitudeDelta: 12.5,
          longitudeDelta: 35,
        }}>
      >
        {bufferedMarkers.map((marker, index) => (
          <Marker key={index} coordinate={marker.coords}>
            <CustomMarker label={marker.id} />
          </Marker>
        ))}
      </MapView>
      {markers.length === 0 && (
        <ReanimatedBubble />
      )}
      <Button
        title="PRESS ME TO CHANGE MARKERS"
        onPress{() => {
          generateNewMarkers();
        }}
      />
    </View>
  )
}

On the iOS simulator with these changes, the map now works as expected!

7. Solution Trade Offs

Solution #1

Removing RNR is surely a less than perfect solution for most developers. Animation is becoming increasingly important as a standard in consumer grade applications, and users now expect it.

I hope to learn more about the root cause of the conflict between markers and RNR, and can hopefully update this post with a more clear answer. Better yet, this issue can probably be resolved within RNR itself, making this issue a bad memory of the past.

With enough trial and error though, you should be able to find work arounds within your animated components rendered with your map to bypass this issue, while keeping those silky smooth user experiences.

Solution #2

Let me first say that my team implemented Solution #1 within our app. We made solution 2 along with solution 1 as PoC's, to then evaluate and decide what would work best for our use case. The example code I gave for solution 2 is valid, however it has some obvious shortcomings (with respect to the code example presented within this blog):

It has a higher time and space complexity⁴
As written, it has a ceiling for the possible amount of marker objects.

Admittedly, I didn't bother optimizing the given example, as it serves its purpose. Some potential improvements could look something like this...

Have the buffer array in the hook match the size of your marker array, such that n_marker === n_buffer. If n_marker ever decreases in length, maintain the longest length of n_buffer by inserting buffer marker objects in place of the absent markers. By doing this, you are not limited with respect to the amount of markers you can have and you're not dropping keys. However, you will have more data persisting, which could lead to performance issues.

Additionally, with the use of useEffect() in the buffer hook, the data being used for the markers will exist outside of a component's typical life cycle. This again has some performance considerations, and you may want to be wary of memory leaks. To mitigate this, you could enhance the hook to manage buffer data when the map component is mounted and unmounted. Rather than digging into this here, I'll point you to this blog post by Caelin Sutch.

If you and your team implement something akin to solution 2, and it is much better than what I have outlined here, please let me know! I can update this blog post⁵, or point readers to any material you may produce.

Regardless of which solution you may choose to implement, if you're using custom markers you're going to want to be mindful of performance. This blog post by Eli Bucher covers performant map pins well, and provides a good solution.

8. TL;DR

If you are getting the error Exception thrown while executing UI block: * -[__NSDictionaryM setObject:forKeyedSubscript:]: key cannot be nil while trying to use custom markers with Google Maps on iOS via react-native-maps, it is possible the root cause is a conflict with react-native-reanimated.

I don't know exactly what the conflict is, however, I have two solutions that may work for you!

Solution #1

If feasible, you can remove react-native-reanimated from your project and that should resolve the issue for you.

If that isn't an option, I would encourage you to rebuild your map in a test project without reanimated, and implement your map so that you are using custom markers (and it should work). Then, reintroduce reanimated into that project, and add your desired functionality in piece wise until you can isolate what method in reanimated is causing the error. In our case, it was our use of FadeInUp and FadeInOut. Your mileage may vary, but you should be able to find alternative animation solutions while preserving your app's behaviour.

Solution #2

Agnostic of any react-native-reanimated conflicts, this error is thrown because the indices from the marker array you're mapping over are being dropped in the case that the marker array length, n, is changing such that n' < n.

You can create a hook that will always return a constant sized array containing marker objects to your map view, so that it may map your objects to custom markers. For every index in this array, there will exist either a real marker object that renders on the map, or a buffer object that will not render on the map. Either way, the indices being used as keys will always exist, therefore you will not hit this error.

9. About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Footnotes

Take that ChatGPT, you can't steal our jobs just yet!
All of the discussion online regarding this issue has been extremely helpful, and I recommend digging into it. While we did not find the answer we were looking for, it provided a solid foundation to dig deeper into the libraries and come up with a solution that worked for us. Thank you to everyone who has contributed to this issue!
These examples are heavily abstracted. I attempted to demonstrate what the broken code looks like in as simple of form as possible.
Asymptotically, both time and space are in the domain of, at minimum, 2n, therefore a O(n) in complexity on the RNR implementation side of things, not considering the internal's of RN or any libraries used. While this doesn't seem bad, anecdotally, our team has noticed RN performance issues which I imagine would show themselves here.
And give due credit, of course!

References and Links

Refresh Token Rotation: What, Why and How?

Taylor Noj — Thu, 17 Nov 2022 16:23:30 +0000

When was the last time you connected an account you own with a third-party application or website? Have you ever used an app marketplace? Have you connected your GitHub account with another website? How many times have you been prompted to “allow access to” or “reauthorize” a connection? It happens so frequently that we barely notice it anymore. What is actually happening behind the scenes during this process? How can we as developers ensure we are safeguarding the applications we build and the people who use them? To start answering some of those questions, let's talk briefly about authorization.

Breaking down the authorization flow

If you recently hit an “Allow Access” button to enable an integration or update an existing app, you’ve likely initialized an authorization flow. The most popular authorization framework, where you are sharing information about your account with a third-party website or app, is known as OAuth 2.0. For example, at Jobber we have an App Marketplace where admin users are able to connect their Jobber accounts to third-party applications. Much like Facebook and Amazon, Jobber also uses OAuth 2.0 to facilitate third-party authorization. The Jobber admin user initiates a third-party app connection, grants requested permissions, and the user is directed to the app. The client request and subsequent authorization involves scopes, which specify and limit the information the third-party application will later be able to access.

But what is actually happening under the hood when you hit “Allow Access”? For an in-depth answer to that question, I suggest exploring the Authorization Code Grant documentation. To broadly summarize, once a user authorizes the client (a third-party website or app), the client receives an authorization code. The client can then redeem this authorization code for an access token and, possibly, a refresh token. The access token is short lived, provides access to the user's resources, and allows a client to make a request to retrieve data through an API. Just how short lived is an access token? This is determined by the authorization server that issues an access token with parameters such as expires_in. When the access token inevitably expires, it can no longer be used to access user data or the API. Attempting to make an API request with an expired access token will result in a 401 Unauthorized error.

What does this mean for the user? If the authorization code grant only resulted in an access token, the user would have to start over and initiate the authorization flow once again. This wouldn’t be a great user experience, but remember that an access token provides access to sensitive information about the user. The longer an access token is valid, the more opportunity for a potential leak of user data. In certain cases, where security is paramount, having the user authorize a website or application for every visit or login could be by design. When it’s not the intended experience, how can developers ensure that a user has to authorize the client only once? Drumroll please. . . refresh tokens!

What are refresh tokens and why should we use them?

Refresh tokens are long-lived credentials that a third-party developer could use to request a new access token after it has expired. To redeem a refresh token, a third-party integration needs to authenticate itself. This is often done using a client identifier and client secret, which are essentially a username and password for the integration.

The ability to request for a ‘refreshed’ access token means that the user can remain logged in without repeating the authorization flow, providing a more seamless user experience. In addition, using a refresh token to request a new access token allows for the authorization server to issue very short-lived access tokens. Stored information should be short-lived whenever possible and an easy way to allow refresh tokens to also be short-lived, is through refresh token rotation.

How should we be using refresh tokens?

The specifics of how refresh token rotation is implemented can vary, but in general the rotation ensures that each time a refresh token is used to request a new access token, the authorization server will return a new access token as well as a new refresh token. When refresh tokens are being rendered invalid more frequently, the risk of replay attacks will decrease significantly.

Third-party developers who are building with Jobber’s Developer Center have the option to enable refresh token rotation. This feature can be selected while creating or when modifying an app. To provide a bit more context, when developers use a refresh token to request a new access token, they are using the following POST request:

POST /api/oauth/token HTTP/1.1
Host: api.getjobber.com
Content-Type: application/x-www-form-urlencoded

client_id=CLIENT_ID&client_secret=CLIENT_SECRET&grant_type=refresh_token&refresh_token=REFRESH_TOKEN

The resources returned are, of course, the new access token and a refresh token. The refresh token will be either the same as before or new, depending on the client's rotation selection.

It is worth noting that there could be instances where the refresh tokens can expire. This may happen when a user has to re-authorize the client after a third-party developer adds additional scopes, or when the admin user chooses to disconnect from the client. This list of possibilities is not exhaustive, however it’s important to know that in cases such as these, the user will need to start the authorization process again if they wish to continue using the integration.

Let’s revisit the authorization flow once again through a Jobber lens:

An admin user decides to connect to a third-party application through Jobber’s App Marketplace.
The authorization flow begins once the Jobber admin user has been redirected to an authorization screen. In this authorization screen, the client is requesting authorization from the admin user who can in turn, approve the request by clicking “Allow Access”.
The client receives an authorization code and then requests an access token and refresh token from the authorization server.
The authorization server returns an access token and a refresh token.
- The access token expires after 60 minutes.
- If refresh token rotation is disabled, the refresh token is long-lived. For any subsequent redemption of a refresh token for an access token, the original refresh token is returned.
- If refresh token rotation is enabled, a completely new refresh token will be issued each time an access token is requested.

Conclusion

The frequency in which we take part in an authorization flow in our day-to-day lives is so high that rarely do we take a minute to ask ourselves, how does this integration actually work? I hope this article serves as a quick introduction into authorization and sparks your curiosity to dig deeper. I also hope that if you are a developer working on an app marketplace or building a third-party application, that you strongly consider utilizing refresh token rotation. Whenever possible, stored information should have a short lifetime and refresh token rotation is an easy way to improve the security of your applications.

About Jobber
Our awesome Jobber technology teams span across Payments, Infrastructure, AI/ML, Business Workflows & Communications. We work on cutting edge & modern tech stacks using React, React Native, Ruby on Rails, & GraphQL.

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

Transitioning from the Healthcare Industry

Kyle Liang — Tue, 08 Nov 2022 17:29:30 +0000

I’m Kyle Liang and I pivoted from a career in the healthcare industry to a role as a Software Engineering Intern at Jobber. In between leaving the healthcare industry and starting the internship, I completed a 3-month Web Development Bootcamp Program. This article outlines my transition from research in healthcare to software at Jobber. I started out working with digital health research and operations, helping support technology-enabled digital health programs and small-to-medium sized enterprises to bring their digital health technologies to the market. While working with these technologies, I grew interested in the technical details. During the COVID-19 pandemic, I decided to transition to a career in software development and took the opportunity to do a web development bootcamp with Lighthouse Labs Inc.

Web Development Bootcamp

Prior to the bootcamp, I had never done any software development which made diving into an intensive bootcamp program with no prior experience terrifying. The program was 12 weeks of full time coding, during which we learned popular languages and frameworks like JavaScript and React. Along with those fundamentals technical skills, the bootcamp instructors mentored us in building working software, career management, and technical interview training. Even with all of that preparation, by the end of the bootcamp, I still doubted myself. With only 3 months of coding experience, how could I compete against someone with a 4 year degree in computer science?

Job Hunting

Even before the bootcamp ended, I began hunting for jobs. I applied to countless companies and was often rejected for my lack of experience. I eventually came across a posting that welcomed bootcamp students: Jobber’s Software Engineer Internship Program. Jobber’s wonderful Talent Acquisition team took me through the full interview process of a phone call, technical assessment, interview, and offer to join the internship program. I never felt lost or unsure of what to expect from Jobber’s interview process, which was the best I’ve experienced in any industry.

Software Engineer Internship Program at Jobber

Jobber welcomed us with an excellent onboarding process and software engineer training. However, I still experienced Imposter Syndrome. I felt like I didn’t have enough experience or knowledge and was nervous to take on any challenging work at the beginning of the internship. Despite my initial concerns, my incredible team supported and mentored me to make sure I gained the confidence that I needed.

Since my internship started, I have worked with 2 scrum teams and have contributed to Jobber’s communications system, its online application, and its new mobile application. My favourite part of the internship so far has been helping people in small businesses succeed while growing as a software engineer. At Jobber, there are always ways to contribute to the product, and endless support, mentorship, and learning opportunities to foster your development throughout the internship.

One of the most impactful pieces of advice that I’ve received during my internship was to not be afraid to raise your hand and volunteer for work, even if you are uncertain about whether it is too difficult. At Jobber, your team is there to help you navigate through any challenges and provide guidance when needed. A supportive environment like this has provided me with the confidence to take on new challenges and get the most out of my internship.

I am grateful to contribute to Jobber’s product and to help small businesses succeed. You are truly able to feel like you are making an impact for the small businesses using Jobber. Even though we may not get to interact with the service providers directly, their feedback lets us know that our hard work is having an impact. I sincerely appreciate Jobber for providing this incredible experience and for giving people like me who have minimal software experience an opportunity.

Conclusion

I had a lot of doubt and uncertainty quitting my job in the healthcare industry to pursue a career in software. I was lucky enough to have the support I needed at every step of the transition, from before I quit my job, to during the bootcamp and my job hunt, to when I first started working at Jobber, and even now. Despite my initial concerns, making the jump to software development is one of the best decisions I have ever made.

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!

The GraphQL N+1 Problem and SQL Window Functions

Nick Boers — Fri, 07 Oct 2022 19:40:36 +0000

A post by Clinton Pahl and Nick Boers, PhD

Introduction
AssociationLoader and its Drawbacks
Background for the SQL Window Loader
Aggregate Functions
Window Functions
Memory-Efficient N+1 Resolution
Introducing the WindowLoader
Conclusion

Introduction

At Jobber, we're constantly evolving a modern GraphQL API to support our Web-based interface, mobile app interfaces, and third-party integrations. GraphQL allows these clients to specify the field structure and data they need in response to API queries. For example, consider the following GraphQL query:

query JobVisits {
  jobs {
    nodes {
      visits {
        nodes {
          title
        }
      }
    }
  }
}

In this GraphQL query, the client has requested the title for each job visit. The response will contain an array of jobs, and for each job, an array of visits with titles. The structure of the JSON response is similar to the query:

{
  "data": {
    "jobs": {
      "nodes": [
        {
          "visits": {
            "nodes": [
              {
                "title": "Initial Assessment of Property"
              },
              ...
            ]
          }
        },
        ...
      ]
    }
  }
}

In our Rails application, we use the popular graphql Ruby gem to resolve GraphQL queries. When used naively, it essentially resolves queries as a depth-first tree traversal, which leads to the N+1 problem in GraphQL.

GraphQL’s N+1 problem, which might be better thought of as the 1+N problem, refers to the number of fetches from a backend data store necessary to resolve a relationship. In the previous example, a single fetch can obtain all of the jobs for an account. After obtaining all of the jobs, a naive resolver fetches the visits for the first job, the visits for the second job, and so on. After one fetch to get all the jobs, N additional fetches get the visits. Given the depth of relationships possible in a GraphQL query, these fetches from the backend data store can quickly balloon and lead to poor performance.

The poor performance will specifically be seen in API response times. Let’s assume the above query returned 100 jobs and that fetching the visits for each job from the database takes 2 ms. In this example, the additional 100 fetches will add 200 ms to the response time for just one field.

`AssociationLoader` and its Drawbacks

For the relationships subject to the N+1 problem, the graphql-batch Ruby gem and its AssociationLoader provide some relief. This gem was developed by Shopify. Using Ruby promises (provided by the promise.rb Ruby gem), it fundamentally alters the order of field resolution for GraphQL queries, and in a sense, converts the query from a depth-first traversal to a breadth-first traversal.

As a breadth-first traversal, a call to resolve visits for a single job doesn’t actually fetch data for the job’s visits. Instead, it returns a promise. The resolver returns a promise for each call to resolve visits. Once they're all batched, the data can be fetched from the backend in a single operation.

Under the hood, the AssociationLoader leverages ::ActiveRecord::Associations::Preloader. Resolving a field with promises involves collecting all of the records (e.g., jobs) where an association (e.g., visits) needs to be resolved. The Active Record Preloader then goes ahead and fetches all of the data in a single data fetch operation. After loading the data, the individual promises are fulfilled using the Active Record data loaded into memory.

Using the AssociationLoader and GraphQL Ruby field extensions, developers can easily configure a field to preload the required associations when it's resolved:

field :visits, Types::VisitType.connection_type, null: false, preload: :visits

Unfortunately, this approach has a weakness as soon as a GraphQL query includes pagination arguments. Consider the following slightly more complicated GraphQL query, which obtains the first three visits for each job.

query FirstThreeJobVisits {
  jobs {
    nodes {
      visits(first: 3) {
        nodes {
          title
        }
      }
    }
  }
}

In this example, the Active Record Preloader can still be used to satisfy the visits association for each job. Suppose jobs contains all of the jobs, resolving visits in a single fetch might involve a call like

::ActiveRecord::Associations::Preloader.new.preload(jobs, :visits)

Unfortunately, this code will fetch all of the visits for each job, even though the GraphQL response will only include the first three visits for each job. In this approach, the Rails application will need to perform the pagination, and in the process, it fetches unnecessary data from the database, which consume unnecessary memory in the Rails application.

Background for the SQL Window Loader

Our initial solution to preloading was based on the AssociationLoader, and it served us well to a point. It addressed the GraphQL N+1 problem, and it significantly reduced our API response times when compared to not addressing the N+1 problem. Unfortunately, as we increased the use of connection types (with their first, last, before, and after arguments), retrieving all associated records from the database only to paginate them in the Rails application was inefficient. For objects with many associated records, it consumes more memory than necessary.

After recognizing the problem, we brainstormed options to offload some of the work onto the database server to ultimately reduce the Rails application’s memory consumption. One particularly promising avenue involved SQL window functions. After deciding to pursue SQL window functions, we started our work by considering the WindowKeyLoader example described in the graphql-batch repository.

Given that many readers may be unfamiliar with SQL window functions, the following subsections provide some background.

Aggregate Functions

An understanding of SQL aggregate functions will help in understanding SQL window functions. Consider a traditional SQL aggregate function such as count(). In a statement involving such an aggregate function, the GROUP BY clause groups records, each distinct group becomes a row in the result. The database management system (DBMS) applies the aggregate function to each group’s records to produce the function’s output.

For example, consider the following query:

SELECT
  account_id,
  count(user_id) AS user_count
FROM
  users
GROUP BY
  account_id

This query

selects records from the users table,
groups those records by account_id and essentially flattens them so each row in the result has a distinct account_id, and
computes the field user_count for each row in the result by applying the function count() to the records in each group.

That (simplified) explanation of the GROUP BY clause provides some background for understanding window functions.

Window Functions

When using a window function, records are conceptually grouped only for the context of the function. Those groups are not flattened in the result. The OVER clause immediately following the function name will apply the function to a window, and the OVER clause itself defines the groups of that window.

For example, consider the following query:

SELECT
  account_id,
  user_id,
  login_count,
  rank() OVER partition_by_account_id AS user_rank
FROM
  users 
WINDOW partition_by_account_id AS (
  PARTITION BY account_id
  ORDER BY
    login_count DESC
)

In this example, the DBMS applies rank() to subsets of the records, where PARTITION BY account_id defines the subsets. For each unique account ID, the result includes a field user_rank with values from 1 to n where n is the number of users for the unique account ID. The users are ranked by the number of times they have logged into the system.

Memory-Efficient N+1 Resolution

Recall the following GraphQL query from earlier, which obtains the first three visits for each job.

query FirstThreeJobVisits {
  jobs {
    nodes {
      visits(first: 3) {
        nodes {
          title
        }
      }
    }
  }
}

If jobs typically have very few visits, the AssociationLoader might be a reasonable solution. If jobs have many visits, it would load many visits from the database that would ultimately be discarded because of the first: 3 filter.

Using SQL window functions, it’s possible to apply the first: 3 filter at the query level and load only the necessary records into the Rails application. For example, the following query would only load the first 3 visits for each job.

SELECT
  numbered_visits.*
FROM (
  SELECT
    *,
    row_number() OVER partition_by_job_id AS row_number
  FROM
    visits
  WHERE
    job_id IN (...)
  WINDOW partition_by_job_id AS (
    PARTITION BY job_id
    ORDER BY id ASC
  )
) AS numbered_visits
WHERE
  row_number <= 3

For direct associations (i.e., those where a foreign key links two tables), deriving this SQL is a rather mechanical process. It’s mechanical enough that we created a new WindowLoader to make use of SQL window functions for resolving these associations.

Introducing the `WindowLoader`

When data access patterns suggest an SQL window function will improve the performance resolving a field, simply adding our new window_load argument to the GraphQL field method will cause the resolver to use SQL window functions when resolving the field. The new window_load argument provides the name of the association, e.g.,

module GraphqlSchema
  module Main
    module Types
      class JobType < GraphqlSchema::Common::Types::BaseObject
        ...   
        field :visits, Types::VisitType.connection_type, window_load: :visits
        ...
      end
    end
  end
end

In our BaseField, derived from the graphql Ruby gem’s Schema::Field, the initializer accepts this window_load argument. When the argument specifies an association, the BaseField constructor adds a custom connection extension to the field (WindowConnectionExtension).

Our WindowConnectionExtension class inherits from the gem’s Schema::Field::ConnectionExtension and Schema::FieldExtension classes. This connection extension has two hooks that wrap field resolution: resolve and after_resolve. The former hook is called to resolve the field, and in this instance, it uses our WindowLoader class to obtain a Ruby promise for the resolution of the field. The latter hook is called after field resolution and after the resolution of promises, and in this instance, it uses our WindowConnection class.

The former class, GraphqlSchema::Common::Loaders::WindowLoader, which inherits from GraphQL::Batch::Loader, first records the foreign keys that will need to be used in the SQL query. In response to .load calls, it returns promises. To finally resolve the promises, it generates and runs a single SQL query that uses the previously-described window functions. Most developers using this window loader are totally unaware these steps occur behind the scenes.

The latter class, GraphqlSchema::Common::Pagination::WindowConnection, which inherits from GraphQL::Pagination::ArrayConnection, produces a result with the expected fields for pagination, e.g., cursors and total counts.

Conclusion

Naively using the graphql Ruby gem to resolve GraphQL queries for a Ruby on Rails API leads to an implementation that suffers from GraphQL’s N+1 problem. Iterating on that solution with a gem such as graphql-batch with its AssociationLoader can dramatically improve the situation by solving the N+1 problem and significantly reducing API response times. When a GraphQL query accepts arguments for pagination, a solution like the AssociationLoader can lead to loading more data than necessary from the database, and as a result, higher than necessary memory consumption in the Rails server. With SQL window functions, it’s possible to offload the pagination onto the database server so that the Rails application does not receive more records than necessary. Given the flexibility of the graphql and graphql-batch gems, it’s possible to create an easy to use interface for loading data using SQL window functions.

About Jobber

If you want to be a part of a collaborative work culture, help small home service businesses scale and create a positive impact on our communities, then visit our careers site to learn more!