grapher

Grapher

This folder contains the code for Grapher, our client side data exploration and visualization library. The Grapher pipeline is explained below.

Step 1: The Grapher Config

The user navigates to a grapher page and the browser fetches the Grapher Config.

The Grapher Config contains 3 main ingredients:

• Where to get the Data and Metadata
• Any Transforms to apply to the data
• What Chart Components to show

Step 2: The Data

Once the Grapher Library has parsed the Grapher Config, it fetches the Data from the URLs in that config (or in some cases the Data is embedded right in the Grapher Config).

The Data is downloaded in two pieces (though technically the second piece is optional):

1. The Data in CSV (or TSV, JSON, etc). For example:

Country,GDP,Year
Iceland,123,2020
France,456,2020
...

2. The Metadata about the Columns in the Data (including source information). For example:

Column,Name,Source
GDP,Gross Domestic Product,World Bank
...

Then Grapher's Table Library parsed the Data into memory as a Table. This Table has Rows and Columns.

The initial Table is called the Root Table.

Step 3: Global Transforms

If the Grapher Config specified any Transforms such as filtering or grouping, the Table Library will apply those.

For example, if a "Min Year Transform" is specified, rows earlier than that year will be filtered.

Step 4: Child Tables

The Grapher Library then derives one Child Table for each Chart Component from the Root Table.

If the author specified different Transforms for different Chart Components—i.e. a different year to show on the Map Component—those are applied.

All Chart Components can now also make any changes they want to their Child Table without affecting other Chart Components. If Transforms are made to the "Root Table", those changes automatically propagate down to all Child Tables.

Step 5: Rendering

Now all the Chart Components have all their own Tables and Grapher renders to the user's screen.

As the user interacts with Chart Controls, changes are made to the respective Tables and the visualizations update.

Flowchart

graph LR
UserVisitsPage((When User Visits Page))
UserVisitsPage --> Load[Load Grapher Config]
Load --> DataNeeds[Determine Data Needs]
DataNeeds --> Data[Download Data]
DataNeeds --> Metadata[Download Metadata]
Data --> RootTable[Make Root Table]
Metadata --> RootTable[Make Root Table]
RootTable --> GlobalTransforms[Apply Global Transforms]
GlobalTransforms --> ChildTable1[Derive Table for Map Chart]
GlobalTransforms --> ChildTable2[Derive Table for Line Chart]
GlobalTransforms --> ChildTableN[Derive Table for ...]
ChildTable1 --> Render
ChildTable2 --> Render
ChildTableN --> Render
Render --> UserEditsTransforms((When User Uses Controls))
UserEditsTransforms --> GlobalTransforms
UserEditsTransforms --> DataNeeds