Forem: Shawn Schwartz

Shipping Carbon: What I Learned About Product Design by Building in the App Store's Most Crowded Category

Shawn Schwartz — Sat, 11 Apr 2026 19:46:46 +0000

A look at the process, trade-offs, and philosophy behind building an app from scratch, and what I learned about constraint, incentive design, and building for real users as a solo developer.

I’m a product data scientist at Slack by day. Carbon is something I built on the side — but the process drew on the same instincts: starting with a real problem, making decisions with limited data, and knowing when to ship.

It’s a task manager — maybe the most crowded category on the App Store — and I designed and built it alone, from concept to launch. This month, I shipped it.

This post isn’t a product announcement. It’s a walkthrough of how I got here: the problems I was trying to solve, the design decisions I made (and the ones I almost made), and what I learned building a consumer app from a blank Xcode project to a live App Store listing.

Starting with the problem, not the solution

I didn’t set out to build a task manager. I set out to fix a specific frustration.

I’d been using Todoist for almost a decade. It’s an amazing app. But over time, my setup had grown into something that actively worked against me — 200+ tasks spread across projects, labels, and priority levels. Every time I opened it, I had to spend mental energy parsing my own system before I could decide what to work on.

The tool designed to reduce cognitive load was creating it.

I started paying attention to when I actually felt productive, and I noticed a pattern: it was when I had a short, clear list for the day and a vague sense of what was coming next. That was it. Not a project plan. Not a prioritization matrix. Just: here’s what I’m doing today and here’s what’s on deck.

That observation became the entire product.

Constraint as a design principle

The first real design decision was inspiration from Ugmonk’s Analog to-do system.

Early sketches had five cards. Then four. I experimented with a “This Week” card, a “Waiting On” card, a “Projects” view. Each one added surface area that pulled the app toward the same complexity I was trying to escape.

Thus, a three-card system: Today, Next, and Someday was the obvious choice. There’s now, there’s soon, and there’s eventually. That’s how most people I’ve spoken to intuitively organize priorities when you ask them to think out loud. Any fewer and you lose useful nuance. Any more and you’re back to managing a system instead of doing work.

The constraint is deliberate. When your Today card can only hold so much, you’re forced to make a choice about what matters. That friction — the moment where you have to decide “is this actually a today thing?” — is the feature.

Three cards. That’s the whole system. Today for what matters now, Next for what’s coming, Someday for everything else. No folders, no labels, no infinite lists.

Rethinking the checkbox

Once I had the card structure, I needed to figure out task states. The obvious approach is a checkbox: done or not done. But that never matched how I experienced work.

A task might be in progress. It might be something I handed to a colleague. It might be something I’m waiting on. Reducing all of that to a binary is like throwing away useful information.

I decided to implement the following five signal states: Pending, In Progress, Completed, Delegated, and Follow Up. The interaction model is a swipe — you cycle through states with a gesture, and each state has a distinct visual treatment so you can scan your list and understand the full picture in a second.

This was a trade-off. Five states is more complex than a checkbox. I spent a lot of time asking whether I was violating my own minimalism principle. What convinced me was testing: when I used the signals for a few weeks, I stopped opening the app to “check” things and started opening it to update things. The list became a living document instead of a static to-do list. That felt like a meaningful difference.

Natural language input and removing friction

I really wanted task entry to feel like writing on a sticky note — not filling out a form with separate fields for title, date, time, and tags.

Carbon parses natural language as you type. “Review deck for Sarah tomorrow at 3pm #work @sarah” becomes a task with a date, time, tag, and delegate already attached. No dropdowns, no modals, no extra taps. Voice capture works the same way.

The parser was one of the more technically interesting parts of the build. Date and time expressions are surprisingly ambiguous in natural language — “next Friday” means different things depending on when you say it, “morning” is vague, and people write dates in a dozen different formats. I spent more time on edge cases in the parser than on most other features combined. But it was worth it. The input experience defines how an app feels to use day-to-day, and reducing a multi-field form to a single line of text meaningfully changes the friction of capturing a thought.

Carbon understands you. Type dates, times, tags, and delegates in plain language — metadata is extracted as you go.

Gestures and physicality

One design goal that’s harder to articulate: I wanted Carbon to feel physical. Like manipulating objects, not operating software.

Swipe right to cycle through signal states. Swipe left to erase. Shake to undo. These aren’t novel interactions individually, but together they create a tactile rhythm. You can blaze through your morning review — swipe, swipe, swipe — without ever tapping a button or opening a menu.

This was inspired by how I interact with physical notebooks. When I cross something off on paper, there’s a satisfying directness to it. I wanted that same sense of immediacy in the app. No confirmation dialogs, no “are you sure?” prompts. Act, and if you made a mistake, shake to undo.

Focus mode: one app, not two

I’ve used the Pomodoro technique for years, and I always had a timer running in a separate app alongside my task manager. It felt like a missing integration — I’m looking at my tasks in one app and tracking my focus in another.

Carbon has a built-in focus timer with customizable work and break intervals and session tracking. It’s simple by design — I didn’t want to build a full-featured time tracker. I wanted the minimum: start a session, do the work, see how your focus patterns develop over weeks.

The decision to include this was driven by a principle I kept returning to: if I find myself switching to another app to complete a workflow that started in Carbon, that’s a gap. Not every gap is worth filling, but this one added real value without adding complexity.

Insights without gamification

Carbon has a section called Insights that shows you patterns in how you work — tasks created vs. completed, weekly rhythms, a 52-week activity heatmap, and gentle nudges when something looks off (like a task that keeps bouncing between cards without resolution).

I was careful here. Productivity analytics can easily become a game that optimizes for the metric instead of the outcome. I didn’t want people trying to “fill in” their heatmap or chasing streak counts.

So the design is intentionally quiet. Muted colors, no scores, no achievements. The heatmap is just a grid — it shows you what happened, and you decide what it means. The nudges are framed as questions (“This task has moved between cards three times — want to break it down or let it go?”) rather than judgments.

The goal is reflection, not optimization.

Insights shows you how you work — not to gamify your productivity, but to help you notice what’s actually happening. Quiet patterns, gentle nudges.

 Download Carbon on the App Store

Importing from existing tools

I knew that asking people to start from scratch was a high barrier. If you’ve spent a year building a system in Todoist, Things, or Notion, you’re not going to abandon it to try something new — even if you’re frustrated with it.

Carbon can import tasks directly from all three. It was a meaningful investment in engineering time, but it fundamentally changes the pitch from “start over with a new app” to “bring what you have and see if a simpler structure works better”.

This was a product decision as much as a technical one. Reducing switching cost is one of the highest-leverage things you can do for adoption, and it signals respect for the user’s existing investment.

The business model question

This is the part I feel most strongly about, and it’s where I’ll get a little opinionated.

The dominant business model for productivity apps right now is subscriptions. $4-8 per month, often with a free tier that’s deliberately limited to push you toward paying. The economics make sense for the companies — recurring revenue is more predictable and valued higher by investors.

But from the user’s perspective, you’re paying $50-100 per year for a to-do list. And if you stop paying, you lose access to your own data or get downgraded to a barely functional free tier.

I decided early on that Carbon would be a one-time purchase. You pay once, you own it, you get every feature. No tiers, no annual renewals, no “pro” unlock.

This wasn’t just a philosophical stance — it was a design constraint that shaped the product. When you’re not optimizing for monthly active users or engagement metrics, you make different decisions. You don’t need push notification strategies to pull people back in. You don’t need to gate features behind a paywall. You just need to make something good enough that people feel it was worth the price.

I can sustain this model because I’m one person with low overhead. No investors, no office, no team to pay. That’s not a limitation — it’s a structural advantage that lets me price the app at what I think is fair.

And this isn’t a “ship it and move on” situation. I’m actively building Carbon based on user feedback. The app will keep getting better — new features, new platforms, continuous improvements — and it’s all included. What you buy today is the starting point, not the ceiling.

Privacy as a non-negotiable

Task data is inherently personal. It’s a map of your priorities, your commitments, your anxieties. I didn’t want to be in the business of knowing any of that.

Carbon encrypts data in transit and at rest. There’s no analytics on your tasks. No usage data sold to third parties. No ads. This wasn’t a competitive differentiator I reverse-engineered from market research. It was a baseline requirement that I would want from any app I put my personal data into.

What I’d tell other solo builders

A few things I learned building Carbon that I’d want to pass along:

Start with your own pain. The best early product decisions came from the fact that I was building something I actually needed. When you’re your own user, you have a bullshit detector that no amount of user research can replicate.

Constraint breeds clarity. Every time I was stuck on a design decision, the answer usually involved removing something, not adding it. The three-card system only works because I said no to the fourth card. The signal states only work because I said no to the sixth one.

Ship before it feels ready. Carbon isn’t perfect. There are features I want to add, edges I want to polish. But the version on the App Store today is useful, and the feedback I’m getting from real users is more valuable than another month of building in isolation.

Your business model is a design decision. How you charge shapes what you build. The subscription model has real downstream effects on product incentives that most people don’t think about until they’re deep in it.

What’s next

Carbon is live on iPhone and iPad today. A companion web app is also available, and the native Mac app is currently under review. I’m building in public and planning based on what users tell me matters most.

Carbon syncs seamlessly across every device. Start on your phone, finish on your desktop, check in from any browser.

If any of this resonated — whether you’re interested in the app or just the process — I’d love to hear from you.

 Download Carbon on the App Store

Download Carbon on the App Store | carbonapp.co

Welcome to my Newsletter, Sustained Attention

Shawn Schwartz — Mon, 05 Jan 2026 00:00:00 +0000

I'm Shawn — software engineer turned computational neuroscientist turned data scientist. I started writing code at 12, shipped iOS apps through high school, and somehow ended up studying how the brain guides our learning and attention for my PhD at Stanford.

These days I work across both worlds. At Slack, I built ML systems for end user archetypes and data pipelines that process millions of data points. In the lab, I run experiments measuring pupil dilation and brain activity to understand why we remember some things and forget others. Along the way, I've published research in journals like Current Directions in Psychological Science, and built and shipped production-grade open-source tools like eyeris — an opinonated pupillometry preprocessing R package now used by many researchers across the globe.

This blog is where those threads converge. Expect posts on:

Signal processing — the neural kind (EEG, pupillometry) and the data engineering kind (streaming, pipelines)
Building research tools — API design, reproducibility, and making software that scientists actually want to use
Attention and memory — what cognitive neuroscience tells us about focus, distraction, and learning

I'm kicking things off with two pieces:

Attending to Remember — how the quality of your attention — down to the millisecond — influences whether and how you'll remember
Engineering a High-Performance, Opinionated Pupillometry Preprocessing Framework in R — 3,500+ Downloads Later: The Engineering Behind eyeris

If you're interested in brains, data, or building things that work with human cognition instead of against it — stick around. You can also find these posts on my blog, Medium, and my Substack, Sustained Attention.

– Shawn

Engineering a High-Performance, Opinionated Pupillometry Preprocessing Framework in R

Shawn Schwartz — Thu, 01 Jan 2026 00:00:00 +0000

3,500+ Downloads Later: The Engineering Behind the eyeris R package

eyeris[1][2] – a CRAN-published R package for reproducible pupillometry preprocessing with BIDS compliance, automated QC reports, and scalable pipeline tools.

Building for Reproducibility and Scale

When designing eyeris, I faced a unique engineering challenge: create a system that's both user-friendly for researchers with minimal programming experience, yet powerful and extensible enough for advanced computational users. Here's how I solved several key technical problems.

Architecture: The Pipeline Pattern

Design Philosophy

The core architectural pattern in eyeris is a functional pipeline where each preprocessing operation:

Takes an eyeris object as input

Applies a transformation to the pupil timeseries data

Records its parameters and provenance

Returns the modified eyeris object

This enables both pipe-based workflows (%>% or |>) and function composition while maintaining full traceability.

Figure 1: Diagram showing eyeris object structure.

The `eyeris` Object Structure

eyeris_object <- list(
  timeseries = list(
    block_1 = data.frame(...),   # Columnar timeseries with transformations
    block_2 = data.frame(...)
  ),
  events = data.frame(...),      # Event markers from experiment
  blinks = data.frame(...),      # Detected blink intervals
  params = list(...),            # Full parameter history
  latest = "pupil_raw_deblink_detransient_...",  # Current column name
  metadata = list(...)           # Sampling rate, eye tracked, etc.
)

The key innovation is the progressive column naming scheme. Each operation appends its name to the previous column:

pupil_raw
pupil_raw_deblink
pupil_raw_deblink_detransient
pupil_raw_deblink_detransient_interpolate
...

This enables:

Full lineage tracking at the data level
Easy comparison between processing steps
Rollback to any intermediate state
Automatic figure generation showing progressive corrections

Signal Processing Implementation

1. Blink Artifact Removal

Blinks are the primary artifact in pupillometry. My implementation:

deblink <- function(eyeris, extend = 50, ...) {
  pipeline_handler(
    eyeris,
    operation = deblink_pupil,
    new_suffix = "deblink",
    extend = extend,
    ...
  )
}

(https://github.com/shawntz/eyeris/blob/dev/R/pipeline-handler.R)

The pipeline_handler() is the "secret sauce" –– it handles all the boilerplate:

Iterates over blocks
Manages column creation and naming
Records parameters
Updates object state
Logs progress

Figure 2: Before/after plot showing blink removal and interpolation.

2. Lowpass Filtering with Visual Inspection

For filtering, I implemented Butterworth filters using gsignal[3] with an optional frequency response visualization:

#' @param wp The end of passband frequency in Hz (desired lowpass cutoff).
#' @param ws The start of stopband frequency in Hz (required lowpass cutoff).
#' @param rp Required maximal ripple within passband in dB.
#' @param rs Required minimal attenuation within stopband in dB.
lpfilt <- function(eyeris, wp = 4, ws = 8, rp = 1, rs = 35, plot_freqz = TRUE, ...) {
  # design a Butterworth filter with minimum order to meet requirements
  fs_nq <- fs / 2
  foo <- gsignal::buttord(wp / fs_nq, ws / fs_nq, rp, rs)  # gsignal package
  filt <- gsignal::butter(foo, output = "Sos")

  # users can inspect filter characteristics before applying
  if (plot_freqz) {
    # generate Bode plot
  }

  # filter twice (forward and backward) to preserve phase information
  gsignal::filtfilt(filt, prev_pupil)
}

(https://github.com/shawntz/eyeris/blob/dev/R/pipeline-lpfilt.R)

This transparency helps researchers understand exactly how the filter affects their data's frequency content.

Figure 3: Frequency response plot (Bode diagram) of the Butterworth filter.

Performance Optimization: Database Storage

The CSV Problem

Most recently, I ran a pupillometry study with 82 participants, 6 sessions each, which generated approximately 800GB of derived data:

~500 timeseries CSV files
~500 confound CSV files
~500 epoch CSV files
~500 event files
100,000+ trial-level diagnostic figures

On cloud storage (S3, GCS), this becomes a nightmare:

High I/O costs for small file operations
Slow syncing
Expensive data transfers

The DuckDB Solution

I implemented an optional database backend using DuckDB[4], an embedded analytical database:

# load your EyeLink eye-tracking data ASC file
eyeris_data <- load_asc("path/to/your/data.asc")

# preprocess and epoch your data with eyeris glassbox functions
processed_data <- eyeris_data %>%
  glassbox() %>%
  epoch(
    events = "TRIAL_START_{trial_type}_{trial_number}",
    limits = c(-0.5, 2.0),
    label = "trial_epochs"
  )

# enable eyeris database alongside CSV files
bidsify(
  processed_data,
  bids_dir = "~/my_eyetracking_study",
  participant_id = "001",
  session_num = "01",
  task_name = "memory_task",
  csv_enabled = FALSE,   # disable generation of CSV files
  db_enabled = TRUE,     # while also creating your eyeris project database
  db_path = "study_database"  # creates study_database.eyerisdb
)

(https://shawnschwartz.com/eyeris/articles/database-guide.html)

Key technical decisions

Schema Design: Separate tables for each data type with proper foreign keys

Batch Inserts: Use DBI's[5] parameterized queries for safe, fast writes

Lazy Loading: Extract only requested subjects/sessions via SQL queries

Compression: DuckDB's columnar format provides ~3-5x compression vs CSV

Implementation Details

eyeris_db_collect <- function(
  bids_dir,
  db_path = "my-project",
  subjects = NULL,
  data_types = NULL,
  sessions = NULL,
  tasks = NULL,
  epoch_labels = NULL,
  eye_suffixes = NULL,
  verbose = TRUE
) {
  # connect to database
  log_info("Connecting to eyeris database...", verbose = verbose)

  # first check if duckdb is installed
  if (!check_duckdb()) {
    log_error(
      "DuckDB is required for this feature. See installation instructions above.",
      verbose = TRUE
    )
  }

  con <- tryCatch(
    {
      eyeris_db_connect(bids_dir, db_path)
    },
    error = function(e) {
      log_error("Failed to connect to database: {e$message}")
    }
  )

  # ensure disconnection on exit
  on.exit(eyeris_db_disconnect(con))

  # get available tables
  all_tables <- eyeris_db_list_tables(con)

  if (length(all_tables) == 0) {
    log_warn("No tables found in database", verbose = TRUE)
    return(list())
  }

  log_info("Found {length(all_tables)} tables in database", verbose = verbose)

  # define all possible data types
  all_data_types <- c(
    "blinks",
    "events",
    "timeseries",
    "epochs",
    "epoch_summary",
    "run_confounds",
    "confounds_events",
    "confounds_summary"
  )

  # use all data types if none specified
  if (is.null(data_types)) {
    data_types <- all_data_types
  } else {
    # validate specified data types
    invalid_types <- setdiff(data_types, all_data_types)
    if (length(invalid_types) > 0) {
      log_warn(
        "Invalid data types ignored: {paste(invalid_types, collapse = ', ')}",
        verbose = TRUE
      )
      data_types <- intersect(data_types, all_data_types)
    }
  }

  log_info(
    "Extracting data types: {paste(data_types, collapse = ', ')}",
    verbose = verbose
  )

  # extract data for each type
  result_list <- list()

  for (data_type in data_types) {
    log_info("Processing {data_type}...", verbose = verbose)

    tryCatch(
      {
        # handle epoch-specific data types
        if (
          data_type %in% c("epochs", "confounds_events", "confounds_summary")
        ) {
          if (is.null(epoch_labels)) {
            # get all available epoch labels for this data type
            type_tables <- all_tables[grepl(
              paste0("^", data_type, "_"),
              all_tables
            )]
            if (length(type_tables) > 0) {
              # extract unique epoch labels from table names
              # handles both eye-L/eye-R and eyeL/eyeR formats
              epoch_pattern <- paste0(
                data_type,
                "_[^_]+_[^_]+_[^_]+_[^_]+_(.+?)(?:_eye[LR]|_eye-[LR])?$"
              )
              extracted_labels <- unique(gsub(
                epoch_pattern,
                "\\1",
                type_tables
              ))
              # remove failed matches (when pattern doesn't match, return original string)
              extracted_labels <- extracted_labels[
                extracted_labels != type_tables &
                  !is.na(extracted_labels) &
                  extracted_labels != ""
              ]
              epoch_labels_to_use <- if (length(extracted_labels) > 0) {
                extracted_labels
              } else {
                NULL
              }
            } else {
              epoch_labels_to_use <- NULL
            }
          } else {
            epoch_labels_to_use <- epoch_labels
          }

          # extract data for each epoch label
          epoch_data_list <- list()
          if (!is.null(epoch_labels_to_use)) {
            for (epoch_label in epoch_labels_to_use) {
              epoch_data <- eyeris_db_read(
                con = con,
                data_type = data_type,
                subject = subjects,
                session = sessions,
                task = tasks,
                epoch_label = epoch_label,
                eye_suffix = eye_suffixes
              )
              if (!is.null(epoch_data) && nrow(epoch_data) > 0) {
                epoch_data_list[[epoch_label]] <- epoch_data
              }
            }
          }

          # combine all epoch data
          if (length(epoch_data_list) > 0) {
            combined_data <- do.call(rbind, epoch_data_list)
            result_list[[data_type]] <- combined_data
          }
          # check if there are any tables for this data type at all
          type_tables <- all_tables[grepl(
            paste0("^", data_type, "_"),
            all_tables
          )]
          if (length(type_tables) == 0) {
            log_warn(
              "No tables found for data type: {data_type}",
              verbose = verbose
            )
          }
        } else {
          # handle non-epoch data types
          data <- eyeris_db_read(
            con = con,
            data_type = data_type,
            subject = subjects,
            session = sessions,
            task = tasks,
            eye_suffix = eye_suffixes
          )

          if (!is.null(data) && nrow(data) > 0) {
            result_list[[data_type]] <- data
          }
          # check if there are any tables for this data type at all
          type_tables <- all_tables[grepl(
            paste0("^", data_type, "_"),
            all_tables
          )]
          if (length(type_tables) == 0) {
            log_warn(
              "No tables found for data type: {data_type}",
              verbose = verbose
            )
          }
        }
      },
      error = function(e) {
        log_warn(
          "Failed to extract {data_type}: {e$message}",
          verbose = verbose
        )
      }
    )
  }

  # filter out empty results
  result_list <- result_list[lengths(result_list) > 0]

  log_success(
    "Successfully extracted {length(result_list)} data types",
    verbose = verbose
  )
  for (dtype in names(result_list)) {
    n_rows <- nrow(result_list[[dtype]])
    n_subjects <- length(unique(result_list[[dtype]]$subject_id))
    log_info(
      "  {dtype}: {n_rows} rows across {n_subjects} subjects",
      verbose = verbose
    )
  }

  return(result_list)
}

(https://github.com/shawntz/eyeris/blob/dev/R/utils-database.R)

This reduces memory footprint by 10-100x for large studies since you only load what you need.

Data Quality Visualization

Interactive HTML Reports

Using rmarkdown[6] and custom plotting functions, eyeris generates comprehensive HTML reports for each preprocessing run. The technical challenge was making these:

Self-contained: All figures embedded as base64

Fast to generate: Parallel figure creation where possible

Interactive: Clickable sections, collapsible details

Figure 4: Full HTML report screenshot showing table of contents, preprocessing steps, and embedded plots.

Gaze Heatmaps

I implemented 2D kernel density estimation using MASS[7] for gaze position visualization:

plot_gaze_heatmap <- function(eyeris, block = 1, bins = 100, ...) {
  # Extract x,y gaze coordinates
  # Compute 2D KDE using MASS::kde2d (MASS package)
  # Overlay on screen dimensions
  # Render with viridis (or another requested) color scale
}

https://github.com/shawntz/eyeris/blob/dev/R/plot.eyeris.R

This helps researchers quickly identify:

Poor calibration
Restricted viewing patterns
Data quality issues

Figure 5: Gaze heatmap showing distribution of eye positions across screen.

Extensibility: The Custom Pipeline System

The Challenge

Users needed to be able to:

Write custom preprocessing steps
Have them automatically integrate with the pipeline
Maintain all provenance tracking
Generate appropriate visualizations

The Solution: `pipeline_handler()`

I created a higher-order function that wraps user operations:

pipeline_handler <- function(
  eyeris,
  operation,
  new_suffix,
  ...
) {
  # 1. Validate eyeris object
  # 2. Get previous operation name
  # 3. Apply core operation (func) to each block
  # 4. Create new column with updated name
  # 5. Record parameters
  # 6. Update latest pointer
  # 7. Generate plots if plot_fun provided
  # 8. Return modified object
}

Users can now write custom extensions with minimal boilerplate:

#' Winsorize pupil values
#'
#' Applies winsorization to extreme pupil values within each block.
#'
#' @param eyeris An `eyeris` object created by [load_asc()].
#' @param lower Lower quantile threshold. Default is 0.01.
#' @param upper Upper quantile threshold. Default is 0.99.
#' @param call_info A list of call information and parameters. If not provided,
#'   it will be generated from the function call.
#'
#' @return Updated `eyeris` object with new winsorized pupil column.
winsorize <- function(eyeris, lower = 0.01, upper = 0.99, call_info = NULL) {
  # create call_info if not provided
  call_info <- if (is.null(call_info)) {
    list(
      call_stack = match.call(),
      parameters = list(lower = lower, upper = upper)
    )
  } else {
    call_info
  }

  # handle binocular objects
  if (is_binocular_object(eyeris)) {
    # process left and right eyes independently
    left_result <- eyeris$left |>
      pipeline_handler(
        winsorize_pupil,
        "winsorize",
        lower = lower,
        upper = upper,
        call_info = call_info
      )

    right_result <- eyeris$right |>
      pipeline_handler(
        winsorize_pupil,
        "winsorize",
        lower = lower,
        upper = upper,
        call_info = call_info
      )

    # return combined structure
    list_out <- list(
      left = left_result,
      right = right_result,
      original_file = eyeris$original_file,
      raw_binocular_object = eyeris$raw_binocular_object
    )

    class(list_out) <- "eyeris"

    return(list_out)
  } else {
    # regular eyeris object, process normally
    eyeris |>
      pipeline_handler(
        winsorize_pupil,
        "winsorize",
        lower = lower,
        upper = upper,
        call_info = call_info
      )
  }
}

(https://shawnschwartz.com/eyeris/articles/custom-extensions.html)

CI/CD and Testing

Automated Quality Assurance

The package includes:

Automated builds on multiple R versions (R-release, R-devel)
Cross-platform testing (Ubuntu, macOS, Windows)
Code formatting checks using custom AIR[8] format rules
Spell checking for documentation
pkgdown[9] deployment for automatic documentation site updates

Figure 6: GitHub Actions workflow status badges.

Test Coverage

eyeris leverages an extensive test suite using testthat[10]. I require all tests to pass before integrating a PR/repo edit upstream. Here's what an eyeris test looks like:

test_that("bin() validates time series monotonicity", {
  # create test data with valid monotonically increasing time series
  valid_data <- data.frame(
    time_orig = seq(0, 10000, by = 100), # original time in ms
    time_secs = seq(0, 10, by = 0.1), # converted time in seconds
    time_scaled = seq(0, 10, by = 0.1), # scaled time
    block = rep(1, 101), # block identifier
    pupil_raw = rnorm(101),
    stringsAsFactors = FALSE
  )

  # create mock eyeris object for testing
  mock_eyeris <- list(
    timeseries = list(block_1 = valid_data),
    latest = list(block_1 = "pupil_raw"),
    info = list(sample.rate = 10)
  )
  class(mock_eyeris) <- "eyeris"

  # test that valid data passes
  expect_no_error({
    result <- eyeris::bin(mock_eyeris, bins_per_second = 5, method = "mean")
  })

  # test with non-monotonic time series (should fail)
  non_monotonic_data <- data.frame(
    time_orig = c(0, 1000, 500, 2000, 3000), # decreasing at index 3
    time_secs = c(0, 1, 0.5, 2, 3), # decreasing at index 3
    time_scaled = c(0, 1, 0.5, 2, 3), # decreasing at index 3
    block = rep(1, 5),
    pupil_raw = rnorm(5),
    stringsAsFactors = FALSE
  )

  mock_eyeris_bad <- list(
    timeseries = list(block_1 = non_monotonic_data),
    latest = list(block_1 = "pupil_raw"),
    info = list(sample.rate = 10)
  )
  class(mock_eyeris_bad) <- "eyeris"

  expect_error(
    {
      eyeris::bin(mock_eyeris_bad, bins_per_second = 5, method = "mean")
    },
    "Time series is not monotonically increasing"
  )

  # test the check_time_monotonic function directly
  # test with empty time vector (should fail)
  expect_error(
    {
      eyeris:::check_time_monotonic(numeric(0))
    },
    "Time vector is NULL or empty"
  )

  # test with single time point (should fail)
  expect_error(
    {
      eyeris:::check_time_monotonic(1)
    },
    "Insufficient non-NA time points to validate monotonicity"
  )

  # test with all NA time values (should fail)
  expect_error(
    {
      eyeris:::check_time_monotonic(rep(NA_real_, 5))
    },
    "Insufficient non-NA time points to validate monotonicity"
  )

  # test with some NA values but valid monotonic sequence
  expect_no_error({
    eyeris:::check_time_monotonic(c(0, NA, 1, 2, NA, 3))
  })

  # test with valid monotonically increasing sequence
  expect_no_error({
    eyeris:::check_time_monotonic(c(0, 1, 2, 3, 4, 5))
  })

  # test with non-monotonic sequence
  expect_error(
    {
      eyeris:::check_time_monotonic(c(0, 1, 0.5, 2, 3))
    },
    "Time series is not monotonically increasing"
  )
})

(https://github.com/shawntz/eyeris/blob/dev/tests/testthat/test-time_monotonicity.R)

Key Engineering Lessons

1. Abstraction Layers Enable Flexibility

The pipeline_handler() abstraction means I can change internal implementations without breaking user code.

2. Columnar Storage >> Row-Based for Timeseries

DuckDB's columnar format is perfect for pupillometry data where analyses typically operate on entire columns.

3. Provenance Tracking Isn't Optional

In scientific computing, you must be able to recreate any result. Automatically recording all parameters and operations was crucial.

4. Performance Matters in Research Tools

Researchers may process hundreds of subjects. A 10-second improvement per subject saves hours in production.

5. Documentation Drives Adoption

We invested in 8+ comprehensive vignettes, function documentation, and tutorials. This was as important as the code itself.

Performance Benchmarks

Typical processing for one participant (1000 Hz data, 30-minute session):

Full pipeline: ~2-3 seconds
BIDS export (CSV): ~1 second
BIDS export (Database): ~0.3 seconds
HTML report generation: ~5 seconds

Database vs CSV for 100 participants:

CSV: 400+ files, ~2 GB total, 15-20 seconds to load all subjects
Database: 1 file, ~500 MB, 2-3 seconds to load all subjects (5-7x faster)

Tech Stack

Core: R (>= 4.1)
Signal Processing: gsignal (Butterworth filters), custom implementations
Data Manipulation: dplyr, data.table, purrr[11]
Database: DuckDB via DBI interface
Visualization: Base R graphics, viridis[12] color scales, custom themes
Reporting: rmarkdown, knitr
File I/O: eyelinker[13] for ASC parsing, optional arrow[14] for Parquet support

Future Technical Roadmap

Parallel Processing: Implement future[15] / furrr[16] for multi-core batch processing

Real-time Preview: Shiny[17] dashboard for live pipeline configuration

Python Bridge: reticulate[18] integration for Python users [in progress]

Distributed Computing: Apache Spark[19] backend for massive multi-site studies

Explore the Codebase & Research Paper

eyeris –» Package: CRAN

eyeris –» Source: GitHub

eyeris –» Docs: shawnschwartz.com/eyeris

eyeris –» Paper: bioRxiv

References

[1] Schwartz, S.T. (2025). eyeris: Flexible, Extensible, & Reproducible Pupillometry Preprocessing. Retrieved from https://cran.r-project.org/package=eyeris

[2] Schwartz, S.T., et al. (2025). eyeris: A flexible, extensible, and reproducible pupillometry preprocessing framework in R. bioRxiv. Retrieved from https://doi.org/10.1101/2025.06.01.657312

[3] van Boxtel, G. (2021). gsignal: Signal Processing. Retrieved from https://cran.r-project.org/package=gsignal

[4] Mühleisen, H. & Raasveldt, M. (2026). duckdb: DBI Package for the DuckDB Database Management System. https://doi.org/https://r.duckdb.org/

[5] R Special Interest Group on Databases (2024). DBI: R Database Interface. Retrieved from https://cran.r-project.org/package=DBI

[6] Allaire, J.J., et al. (2025). rmarkdown: Dynamic Documents for R. Retrieved from https://cran.r-project.org/package=rmarkdown

[7] Venables, W.N. & Ripley, B.D. (2002). Modern Applied Statistics with S. Retrieved from https://www.stats.ox.ac.uk/pub/MASS4/

[8] Vaughan, D., & Henry, L. (2025). Air, an extremely fast R formatter. Retrieved from https://tidyverse.org/blog/2025/02/air/

[9] Wickham, H., et al. (2025). pkgdown: Make Static HTML Documentation for a Package. Retrieved from https://cran.r-project.org/package=pkgdown

[10] Wickham, H. (2011). testthat: Get Started with Testing. The R Journal, 3, 5-10. Retrieved from https://journal.r-project.org/articles/RJ-2011-002/

[11] Wickham, H., et al. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. Retrieved from https://cran.r-project.org/package=tidyverse

[12] Garnier, S., et al. (2024). viridis: Colorblind-Friendly Color Maps for R. Retrieved from https://cran.r-project.org/package=viridis

[13] Barthelme, S. (2021). eyelinker: Import ASC Files from EyeLink Eye Trackers. Retrieved from https://cran.r-project.org/package=eyelinker

[14] Richardson, N., et al. (2024). arrow: Integration to Apache Arrow. Retrieved from https://cran.r-project.org/package=arrow

[15] Bengtsson, H. (2021). A Unifying Framework for Parallel and Distributed Processing in R using Futures. The R Journal, 13(2), 273-291.

[16] Vaughan, D. & Dancho, M. (2022). furrr: Apply Mapping Functions in Parallel using Futures. Retrieved from https://cran.r-project.org/package=furrr

[17] Chang, W., et al. (2024). shiny: Web Application Framework for R. Retrieved from https://cran.r-project.org/package=shiny

[18] Kalinowski, T., et al. (2024). reticulate: Interface to Python. Retrieved from https://cran.r-project.org/package=reticulate

[19] Zaharia, M., et al. (2016). Apache Spark: A Unified Engine for Big Data Processing. Communications of the ACM, 59(11), 56-65. https://doi.org/10.1145/2934664