Forem: Andreas Kollegger

Observable Graphs

Andreas Kollegger — Thu, 21 Jan 2021 11:49:41 +0000

Create graphs with `gram` then share them on ObservableHQ.

Gram makes writing graphs as easy as (a)-->(b)<--(c). But then what? Well, graphs are eminently visual data structures. Naturally, we should visualize them with d3.js. There's a convenient integration called d3-gram which we'll exercise on ObservableHQ.

Observe as I demonstrate...

To follow along you'll need to login or create an account on ObservableHQ. Conveniently, you can use github, google or twitter to sign in.

First, we'll create a notebook:

Click the New button in the upper-right corner
Give it a nice title like # Observable Graph by ABK

Next, import the graphOf function which parses gram then renders a graph.

Create a new block using the + button
Copy/paste the following code
Hit shift-return or press the play/run button ▶️ to perform the import

import {graphOf} from "@akollegger/graph-input"

Now creating a graph visualization is as easy as calling the graphOf function, passing in a Cypher-like gram pattern. Add a new block try this:

abc = graphOf("(a)-->(b)<--(c)");

Neat, right? We can tweak the visualization a little by adding a zoom level and viewbox height:

abc = graphOf("(a)-->(b)<--(c)", {zoom:4, height: 60})

Observe the graph

What is abc? The graphOf function returns an observable view, an input element with a current value. That's right, you can use the graph visualization like a slider or a text input field.

Modify the graphOf block, prepending viewof:

viewof abc = graphOf("(a)-->(b)<--(c)", {zoom:4, height: 60})

Add another block which only contains abc:

abc

When you run that block, notice the result is an object. That's the current value of the input element. It contains an array of all nodes, an array of links, and an array of the current selection. Shift-clicking the graph adds multiple nodes to the selected array.

Object {
  nodes: Array(3) [Object, Object, Object]
  links: Array(2) [Object, Object]
  selected: Array(1) [Object]
}

The array elements are objects that mix d3-js properties for things like x,y coordinates along with the original graph data. Let's add another block to focus on the graph data:

selectedNode = abc.selected.length == 0 ? {} : Object.getPrototypeOf(abc.selected[0])

That's better. Now we can add more details to the graph, which will influence the graph visualization and show up in the selectedNode.

Try this:

viewof abc = graphOf("(a:Person {name:"ABK"})-->(b)<--(c)", {zoom:4, height: 60})

With all this in place, you can create your own graphs by editing the gram string passed into graphOf(). Or you could add a text field using form-input. Or load from an attached file or URL.

For fun, I re-created wikipedia's Gallery of Named Graphs.

A gram of graphs goes a long way. Have fun! :)

Footnotes:

^{[1] Observable Graph by ABK The complete notebook described in this post.}
^{[2] Gram of Graphs - a collection of ObservableHQ notebooks about gram and graphs}
^{[2] @akollegger/graph-input - the particular ObservableHQ notebook which provides the utility functions used in this post}

Gram: a data graph format

Andreas Kollegger — Tue, 19 Jan 2021 12:41:21 +0000

Gram is a textual format for data. We have CSV for tables, JSON for documents, and gram for data graphs.

Use gram when [a,b,c] becomes (a)-->(b)<--(c).

Why another data format?

Anything worth talking about is worth inventing a language for.

-- probably not L. Wittgenstein

Graphs are a powerful and increasingly popular way to think about and work with data. ISO has established a project to formalize an
international standard Graph Query Language. ISO GQL will become the "language of graphs".

But graph technology includes databases, analytics libraries, visualization software, and a host of other tools.

How do we use these things together? We need a common format to connect tools when performing the usual operations:

import/export data file
send/receive network transmissions
copy & paste across applications

We all expect these actions to just work. It's frustrating when any of them are not possible. Tools with limited interoperability feel locked-in and isolated. That's not the graph way.

Gram can make it possible for data graphs to be used in normal everyday work.

Data graphs as information architecture

While there are existing graph formats, gram is designed to complement the Property Graph Model defined by ISO GQL.

Gram is interested in information architectures. Data graphs are the means, not the goal. Rather than talking about graphs, gram speaks in graph to talk about other things.

Some motivating examples include:

Association class: How do we describe the relationship between two things? Not just that they are connected, but how they are connected.
Decorator pattern: Can multiple applications share data while reserving space for their own use?
U.S. Route 66: How can we represent path membership and also information about the path itself?
Bill of materials with finished goods: How do we describe both the plan to build stuff and the stuff that has been built?

Gram attempts to address these scenarios while being friendly to read and write by adopting a path-based representation. Paths all the way down, or all the way up with path composition.

Friendly to read and write

Gram is intuitively simple, inspired by Cypher¹ path expressions.

Information starts with a familiar looking data record of nested properties:

({
    name:"Andreas Kollegger", 
    address: {
        city:"Malmö", country:"Sweden"
    }
})

The extra set of parenthesis wrapping the record provides space for an identifier and a set of labels:

(a:Person:Author { name:"Andreas Kollegger", address: { city:"Malmö", country:"Sweden" }})

Data records can be composed into information structures using relationships:

(a)-[:Wrote]->(b:Blog {title:"Gram: a data graph format"})

We can complete this scene with another related record:

(b:Blog)<-[:Read]-(c:Person)

Putting it all together we could write:

(a:Person)-[:Wrote]->(b:Blog)<-[:Read]-(c:Person)
(a:Author { name:"Andreas Kollegger", address: { city:"Malmö", country:"Sweden" }})
(b {title:"Gram: a data graph format"})
(c {when:date`2021-01-21`})

Nice. The first path is readable as a sentence that "a Person Wrote a Blog that was Read by another Person." Notice in this rewrite we accumulated information -- all the a's are the same record.

Like writing prose, you don't have to say everything at once. Paths appear in an ordered stream that merges forward into a final result. In fact, because any path or sequence of paths forms a valid result you can time-travel or page through a graph.

Use `gram` for data graphs

Gram is:

friendly to read and write
composable, sliceable, pageable and streamable
vendor-neutral
a work-in-progress :)

There is a reference implementation called gram-js which is used along with d3.js in a collection of ObservableHQ notebooks. Next up is integration with other Javascript libraries and conversion to/from other formats.

Check it out and let me know what you think!

Footnotes:

^{[1] Cypher Language inspired both ISO GQL and gram with the use of "ASCII art" to draw graph elements. cypher is a graph database query language.}
^{[2] Gram of Graphs - an Observable HQ notebook collection which illustrates how to use gram and provides handy utilities for visualizing graphs.}
^{[3] gram-js - the gram reference implementation, written in TypeScript}

Graphs on Tap

Andreas Kollegger — Thu, 12 Nov 2020 14:02:27 +0000

Neo4j Relate delivers graphs on tap. From the CLI, from your app, or from a local server, you can provision Neo4j databases whenever you'd like to enjoy the clean, refreshing goodness of graphs.

Extracted from Neo4j Desktop

The best frameworks are in my opinion extracted, not envisioned.
-- David Heinemeier Hansson

Neo4j Desktop is a native application which packages up everything you need for working with the Neo4j DBMS.

Neo4j DBMS download and management
Neo4j connection management
tools like Neo4j Browser, Neo4j Bloom and Neo4j ETL

Internally, there is a GraphQL API through which tools may discover available databases, provision new ones and negotiate authentication.

Neo4j Desktop is extensible. New tools can be added, and you can write your own. But it feels like a walled garden. Peering over the hedge, wouldn't it be nice to easily manage Neo4j from the command line, or from VS Code, or even from within your app?

Neo4j Relate extracts the core operational library of Neo4j Desktop then packages that into a CLI tool, a web server, or a generic Electron app. Let's take the CLI for a spin.

Neo4j at your command

Now witness the firepower of this fully armed and operational ~~battle station~~ CLI
-- The Emperor

The @relate/cli package installs the relate command line tool. There is a PR which will make the CLI available without requiring node.js.

For now, try:

npm install -g @relate/cli

Set up a provisioning environment:

relate env:init --name=graphs-on-tap --type=LOCAL

Accept the default settings when prompted. You'll need to provide this semi-secret access code r31473:

Enter the access code you received from applying at https://neo4j.relate.by/invite:r31473
✔ Do you need to enable authentication? (y/N) · false
✔ Do you need to restrict access to the GraphQL API methods? (y/N) · false
✔ Are HTTP consumers required to have an API key? (y/N) · false
Creating environment... done

The authentication mentioned above is for the GraphQL API, which we'll explore in a later post.

Now install a Neo4j DBMS named "hello" into the provisioning environment named "graphs-on-tap". If you drop 4.1.3 from the end, you can pick which version to install:

relate dbms:install -e graphs-on-tap --name hello 4.1.3

Enter new passphrase: will set the password for the admin user named "neo4j":

✔ Enter new passphrase · 
DOWNLOAD PROGRESS [████████████████████████████████████████] 100%
extracting neo4j... done
[b92ada41] hello

Finally, start the DBMS named "hello" within the "graphs-on-tap" environment:

relate dbms:start -e graphs-on-tap hello

Neo4j DBMS should start up, then you'll be able to browse to http://localhost:7474 to use Neo4j Browser. Use the admin user named neo4j and the password you set above.

Convenient, right?

Try relate --help to see the other available commands and options. For instance, relate env:use to set the "current" environment, saving you a little typing.

Next steps

Neo4j Relate is in alpha, evolving quickly with more capabilities and integrations. Stay tuned and reach out to me with any ideas or questions.

In later posts, I'll explore:

e2e testing using relate to do setup & teardown
hosting Browser, Bloom and other tools without needing Neo4j Desktop
spinning up a server to provision DBMSes on demand
writing nest.js modules to use relate as an application server

Related...

neo4j-devtools / relate

Building a Modern Web Application with Neo4j and NestJS

Adam Cowley ・ Jul 2 '20

#neo4j #typescript #nestjs