Changelog

💱 dbt Semantic Layer integration​

Today we’re releasing a powerful, first-class integration with the dbt Semantic Layer.

At the heart of it is a new Metrics Cell: an easy-to-use UI that lets anyone access trusted, governed metrics, without writing any code. Users can specify metrics, dimensions, and time grains, and get back a data frame that they can use to analyze, visualize, and share.

But there’s more – connecting to the dbt Proxy Server unlocks another superpower: the ability to write dbt-flavored SQL, right in Hex. Now you can use refs, macros, and sources directly in your queries.

We’re so excited to unleash the full potential of the work Analytics Engineers do every day, and empower thousands of users to more easily ask and answer questions of data.

Read more about these new features over on the Blog.

Today we’re launching two new capabilities for Hex:

• First-class integrations with popular orchestration tools Airflow and Dagster
• A public API, allowing you to programmatically run Hex projects from other systems

Together, these represent a new set of super powers for integrating Hex in the rest of your stacks, workflows, and systems. We can’t wait to see where you take it!

🐙 Orchestration integrations: Airflow and Dagster​

You can now schedule Hex apps to run as part of an Airflow DAG or Dagster job. This lets you keep apps up-to-date with your latest data syncs, or do ad-hoc runs with custom input values as part of a complex data pipeline.

Here's Friend of Hex, Pedram, with the details:

These orchestrated runs kick off a new run of the app, optionally set the user-facing cache, and return a link to that particular app run (and other metadata) that you can use downstream.

Airflow & Dagster integrations are available to workspaces on the Teams plan or higher.

🤖 Public API​

We’ve also released a public API with all the endpoints you need to build your own version of these integrations, or something totally unique.

There’s four endpoints in v1.0.0: RunProject, GetRunStatus, CancelRun, and GetProjectRuns. Like the orchestration integrations, RunProject can be used both to trigger runs that set the user-facing cache of an app, or to generate custom one-off runs with specific input parameters.

See the full API docs here for more information. We can’t wait to see what you build!

API access is available to workspaces on the Teams plan or higher.

♻️ GitLab sync​

Git sync now works with GitLab, in addition to GitHub. The flow is exactly the same, you can just add GitLab as a new source in addition to GitHub!

For more information on our version control workflows, check out the git sync docs.

Git sync is available to workspaces on the Teams plan or higher.

🏎️ BigQuery Acceleration​

We updated our BigQuery data connector and those extra-large queries you’ve been writing will see some major performance improvements. In our tests, we've seen queries returning 250k rows run ~5x faster and even greater improvements for larger queries.

To get these improvements, you have to toggle on the “Use BigQuery Storage API” option in your data connection’s settings. You’ll also need to grant the "BigQuery Read Session User" role to the Google Cloud service account used for the connection.

👑 Admin workspace asset usage​

Want to delete an old connection but nervous it might break production apps? Admins can now see how many projects reference each Data Connection, Secret, or Github package in a workspace.

Admins can also drill down into a list of those projects, which is extra powerful when coupled with…. dun dun dun….

👀 Revamped Project View Stats​

We’ve made big updates to project view analytics, moving it up to the top bar and improving the “See more” view. You can see all-time view statistics, latest version views and recent trends, and a list of recent unique viewers.

Now you can easily find out if your latest app is a viral success, or if that connection you want to delete is being used by any active apps.

Other improvements​

• We’ve made the markdown emoji picker less excitable, so it only shows up when you actually want to add an emoji and not just anytime you want to add a semicolon.
• Okta and other directory syncs should now happen much more quickly.
• Hex has better support for other languages like Japanese, Arabic, and Cyrillic in our rendered snapshots and images.
• We’ve improved sorting behavior in complex, multi-level pivot cells.
• We’ve improved Table Display cells ability to display pivoted DataFrames.

This round of updates is entirely focused around improving your quality of life while you work in Hex.

Apps now run faster, filter cells are easier to use, and you can drag 'n' drop cells in Notebook view. You can even copy and paste cells between projects.

🏎️ App first run acceleration​

Folks, now that you have heard my story,
Say, boy, hand me over another shot of that booze;
If anyone should ask you,
Tell 'em I've got those Hex project First Run Blues.
- https://www.youtube.com/watch?v=EcXSbCXxGzw

Previously, when you loaded a published app, you might have experienced what we’ve come to call "the first run blues": even though an app’s cached view loads instantly, changing an input parameter for the first time triggers a really slow run.

Basically, Hex apps need to do an entire top-to-bottom run just once. After that complete run, everything feels snappy as the reactive execution engine determines which cells to run and which to leave untouched.

Today, we’ve cured these first run blues— not with a shot of booze, but with "app run acceleration". This enables a new option for your apps to "pre-run on page load" – basically kicking that first run off in the background, to shorten the time for first user interaction.

The end result is snappier-feeling, faster-to-interact-with Hex apps for your users.

This feature is available for workspaces on the Team plan and above.

🐲 Cell Drag ‘n’ Drop​

You can now drag ‘n’ drop cells in projects to reorder your logic.

Every cell has a little handle near the title that you can grab to start dragging. When you’ve dragged it to a good new home, move your pointer between cells until you see the "Move cell" bar appear between them— and drop!

🌪️ In-app filters​

We’ve made some great updates to Filter cells. The biggest change is that Filters can now be exposed to end users of published apps, just like input parameters.

As an author, you can allow users to edit the entire filter configuration (eg: which columns are filtered and what kind of condition is applied), or limit them to only edit the filter values, as shown in this clip:

You’ll also notice that these filters have dropdowns of options, not blank text boxes! We now pick up values from the underlying columns and present them for selection.

Plus, filter drag ‘n’ drop 😎. Why let entire cells have all the fun? You can now drag and drop conditions around within filter cells to easily reorganize complex logic.

✂️ Cross-project copy & paste​

The tried and true cmd+C → cmd+V copy and paste workflow has a new powerup: Cells can be copied and pasted between projects.

This works, well, exactly like you’d expect it to work: Copy some cells in one project, paste them into another, bim bam boom, you’re done.

The trusty old cmd+c shortcut will copy any cells you have selected in command mode, but you can also use the ··· menu on any cell to copy all upstream, downstream, or any connected cells. This makes it really easy to move complex logic between projects!

Other improvements​

• We’ve added support for connecting to Redshift via SSL.

• You can choose to disable the datatype icons on Table Display columns, leaving more room for the column name.

• Changes to chart configuration are now "debounced", meaning they wait a couple milliseconds to see if you’re going to make another change before they start re-running the chart.

• We fixed a bug that caused chart cells to error on null dates.

• We’ve improved our handling of messy data in map cells, letting you plot data without having to remove Nones or cast strings to numbers.

Bumper Crop
[ buhm-per krop ]
noun: an unusually abundant harvest from a particular crop. The term is now most commonly used in a figurative way to refer to a large amount or influx of something (that’s often likened to a harvest).

// Thanks to ideal growing conditions, California had a bumper crop of olives in 2012.
// The stars aligned and Hex had a bumper crop of new features in August 2022.

Put down the Old Farmer’s Almanac and fire up your Hex workspaces, my data agronomists. The corn is tall, the sun is shining, and we’ve got an unusually bountiful harvest of new Hex features for you: App URL parameters, Schema Browser querying shortcuts, a new map region layer, Clickhouse support, new cell styles, chart cell updates, Library UI improvements, and more.

(Still feeling agricultural? Check out this quick Hex app that lets you explore historical crop yields. Spot any good bumper crops?)

🔗 App URL parameters​

You can now generate a “Current inputs” sharing link that contains pre-filled values for input parameters, so you can share a specific view of an app.

These links actually include the input parameter values in-line, so you could generate them dynamically with code (say, to link from a recent sales app to a pre-filled view of a customer 360 app): https://app.hex.tech/hex-public/app/cc3f7ed7-e662-4d42-b36c-fa2fb8ec2740/latest?_area="United+States+of+America"&_crop="Olives"&_show_all_areas=false.

Though it’s possible to edit them manually, proper URL encoding can sometimes be hard to get right. The tried-and-true way to generate a properly formatted input link is to set the desired input values in a published app and use the “Copy link > Current inputs” button.

🔎 Schema Browser query shortcuts​

There’s a handy new context menu on tables in the Schema Browser that lets you easily copy fully qualified table names or a pre-filled select statement that includes all columns.

Why do this instead of a shorter select *? Well, as you inevitably start narrowing the list of columns to return, going back and writing them out by hand becomes a lot more tedious than just deleting unwanted lines in a pre-written query.

We’re working on more updates to the Schema Browser, so reach out with any feedback on what functionality you’d like to see added!

📍 Map area regions​

Maps have some powerful new functionality: In an Area layer, you can now map data from common regions (countries, US states, counties, etc.) without having to provide complex geospatially formatted data. Just bring a column containing simple values (eg: California, Vermont, Arizona) and let the map layer handle the hard part.

We’re adding more map regions— if you have a region you frequently work with that isn’t in the list, shoot us a message and we’ll add it for you!

👨‍🎨 New cell styles​

SQL and Text cells have a brand new coat of paint!

SQL cells have a streamlined UI that makes it easier to set cache options and simplifies switching between Dataframe and Query mode.

Text cells now blend in with the liminal space between cells in Notebook view, making them more visible as section headings and better differentiating between code and text.

Oh, and while we’re talking about Text cells…

📸 Images in Text cells​

Text cells can now display images! Just drag an image inline, or copy and paste one in. This will add the image to the file directory of the project.

PS: This also works in markdown cells! Historically, local images from the project filesystem could not be referenced in Markdown cells, but this is now possible. You can also drag+drop or copy/paste new images into Markdown cells, just like Text cells.

📊 Charts updates​

There’s a bunch of great updates to chart cells that have shipped out recently:

Layer legends

Multi-layer charts now show all series accurately in the legend! To enable this, select “Automatic” from the color picker config.

X and Y offsets

You can create grouped bar charts using the X or Y-offset config. To group a bar chart, select the same column in the Color and X-offset (or Y-offset, for a Y axis grouped chart). This will “offset” the location of each bar based on that value, creating a grouped layout.

Text layer

Text layers let you add labels and annotations anywhere on a chart. Select a column with the text to display, choose the X and Y columns that control label placement, and then fine-tune with the Dx, Dy, Angle, and Radius configs to place text exactly where you want it.

Use this to add more complex labels, or to call out specific points as important. It works great with Filter cells— filter down the dataframe powering the chart to just the records you want to annotate, and then add the filter_output as a new text layer.

🏡 Clickhouse integration​

Everyone’s favorite super fast database now works out of the box with Hex. There’s nothing particular about the configuration— just add a new data connection, provide the usual authentication details, and you’re off writing Clickhouse queries in SQL cells.

🖼️ Screenshots in Library​

The Library now features a screenshot of every app at time of most recent publish or scheduled run. We hope this makes it easier to find the app you’re looking for, before you click.

Unlike many libraries, we encourage you to judge your apps by their screenshots 😉.

Note: This will be empty for previously-published apps but will now populate the next time they are published or run on a schedule.

Other improvements​

• SQL cells in the same project now share one session. This is a mostly invisible change that lets power users set session params and write complex multi-statement or non-read-only queries.

• If you’ve created a large app with lots of outputs, you can now Clear outputs from the project list view to wipe them clean.

• We’ve banished some unnecessary scrollbars on text cells.

• We fixed an issue that was causing lines and their points on chart cells to be misaligned— or as James put it, “missing the point entirely” 😑.

  

• Transformed columns in Pivot cells now use underscores instead of parentheses as suffixes, like field_sum, field_year, etc. This makes them easier to work with downstream and in Writeback cells.

Today we’re introducing a suite of UI-driven “No-Code” cells in Hex: Pivot, Filter, and Writeback (which you met in my last email). These allow everyone to do many of the most common data manipulation tasks without googling how to write writing any code.

These cells can be configured using just the UI, but they all return dataframes that can still be used in downstream code cells. Each of these cells also has a Duplicate as Code button, allowing you to “eject” to code, and edit and update it to their heart’s content.

This means you can use these cells not only as helpful workflow upgrades, but also as templates to help learn new things, or jumping-off points to dive into more complex operations. Read more on the blog!.

🏗️ Pivot cell​

Perhaps the most powerful part of spreadsheet workflows is pivot tables. Now, we’re bringing them to modern data analytics workflows, with a powerful built-in cell:

Pivot cells can pivot, aggregate, and format data without writing any code. They return a pivot_result object that can be used natively in charts and other Hex cells, and can also output a raw .aggregated or .pivoted dataframe.

🌪️ Filter cell​

The name kind of says it all – these filter dataframes. You can add individual filters, or groups, and toggle whether the operation keeps or removes matching rows.

Filter cells are writing SQL behind the scenes, and you can both view the compiled SQL or fully "eject" to a SQL cell anytime you'd like.

♻️ Writeback cell (again)​

We actually introduced these last time, but they are a core part of the no-code suite. After you’ve pivoted, filtered, and otherwise transformed a dataframe, you can now send it back to the database, using an existing database connection that’s already configured.

Other improvements​

• When deactivating a user, you will now be prompted to choose to transfer ownership of their projects to another user.

• We’ve fixed some timezone issues in dataframe SQL and charts.
• We’ve fixed a bug with the Map cell that was causing it to break without showing an error.
• SQL cell errors are now much easier to read!