Changelog

Today we’re introducing three new features that allow you to work with data at any scale: Query mode, Chained SQL, and Python pushdown.

As a bonus, we’re also removing the 5,000 row limit for Chart cells 🎉.

∞ Query mode​

Query mode for SQL cells lets you work with effectively infinitely-sized data – no more worrying about memory overhead or limits.

In Query mode, Hex leaves the data in the warehouse instead of bringing it into the in-memory kernel. It’s not just a limit preview though: like all cells in Hex, Query mode cells still return an object (in this case, a Query, in lovely purple) that can be used downstream and is represented in the DAG, but is just a pointer to the full result set.

When it’s time to build visualizations, or run in-memory operations over the full dataset, just move the SQL cell back into Dataframe mode (which is still the default). Read more in the docs.

⛓ Chained SQL​

Chained SQL lets you reference previous queries with other SQL cells, relying on Hex to automatically combine the queries into common table expressions (CTEs) on the fly.

To use this, just reference any SQL cell result (a Query or Dataframe) in a downstream query using the same data connection. In the background, Hex will build the with...as statements for you, and you can see the fully-compiled query right there in the cell:

These “Chained SQL” queries work great with Query mode, allowing you to do multiple-step operations without pulling each intermediate result into memory. Then, at the end, you can flip the last cell into Dataframe mode and have a result set ready to use.

Chained SQL is complementary to Dataframe SQL, which lets you query any dataframe, including other query results, Pandas dfs, or even CSVs. Read more about when to use which feature in the docs.

🐍 Pushing down Python​

That’s all great for SQL, but what about Python? Well, that’s becoming cloud-native too.

Last week, we introduced support for Snowpark, allowing you to run Python code right in Snowflake. Think of this like Query mode, for Python: Hex is running the operation in the warehouse, working with the data where it lives instead of shuttling it back and forth to an in-memory kernel.

In Hex, this support is truly first class – any existing Snowflake data source can turn on Snowpark, and then with one click create a connection in a project.

You can learn more about Snowpark and Hex on the blog.

💪 Scalable charts​

Chart cells now perform server-side aggregation to let you plot data at effectively infinite scale. You can pass a dataframe of any size into a chart cell and aggregate it for display right from the chart configuration, without worrying about row or data limits.

Here’s an example of building a simple chart on a 900k row dataframe:

Under the hood, we're using VegaFusion to power this!

Other improvements​

• We’ve made lots of upgrades to the Map cell
• We fixed an interesting issue that caused Chart cells to error when a dataframe had duplicate column names
• App editors now have the option to “Refresh cached state” on published apps, so you can manually reset the cache on an app.
  
• The default pandas installation has been updated to 1.4.2

We’ve got some good ones this week: a brand new map cell, project-level search, and user directory sync.

🗺️ Map cell​

We’ve added a map cell to Hex, so you can visualize geospatial data without writing a single line of new code. Find it in the Display cell tab, right next to charts and table display cells.

When you’re digging into a dataset, the last thing you want to do is muddle through confusing syntax for a geospatial Python library you don’t regularly use. You just want to see your data on a beautiful map.

et voila: https://app.hex.tech/hex/app/eb4f9bd1-6b05-414d-adf4-cb1aa55d9cf3/latest.

Feedback request: Maps can get pretty complex, so we’re shipping a more simple cell and will let user feedback guide our enhancements. Please reply and let us know what kinds of mapping features you’d like the map cell to support!

🔍 Project-level search​

If you’re like me and spend a lot of time in VS code, you’ve instinctively tried to cmd+shift+f in Hex before, to search across an entire project. And like me, felt silly when it didn’t work.

Well if you can work up the courage, try it again! From anywhere in a project, you can now hit cmd+shift+f (or use the Search sidebar tab) to search for text throughout the entire project. Cmd+f still works as usual for find+replace within cells.

🔐 User directory sync​

Hex can now sync users and groups directly from Okta, Active Directory, and pretty much every other directory under the enterprise sun.

This means you can easily manage users in bulk, configure complex group-based access policies, and only make user edits in one system.

This is an Enterprise feature. If you’re excited about it, please get in touch to set this up.

Other improvements​

• Pandas-generated pivot tables (or any multi-index dataframe) can now be rendered nicely in table display cells!

  

• SQL cells have a new option to "Infer numeric datatypes", that lets you turn off automatic decimal → float casting when working with very high precision numbers. Learn more in the docs.

• Metric cells (you know, the ones that display a single value in your app) have been renamed to “Single Value” cells (makes sense, right?).

We have some gems for you today: screenshots in scheduled emails, dbt Docs, a beautiful new Notion integration, and full-screen editing mode.

🖼️ Screenshots in scheduled emails​

Scheduled runs in Hex now have a new option: to include screenshots of the app right in the email! We also gave the scheduled run dialog a bit of a makeover 💅.

Note: screenshots can only be sent to people and groups with App User permission or above on an app.

📑 dbt Docs in schema browser​

Our dbt integration just got way better! Now we show all the documentation from your dbt project right in your schema browser, including column descriptions and column-level test information. They even support markdown!

👀 Notion link previews​

In case you missed it, we shipped an awesome improvement to our Notion embeds: Link Previews! You can paste the link to a published app into any Notion doc, as a mention (one-line link) or preview (full interactive embed). Check it out:

You can read more about it on Notion’s blog here.

↔️ Full-width editing mode​

You can now toggle a full-width editing mode! This is great for those of you with r e a l l y w i d e monitors.

Other improvements​

• The "upper left" logo will link to Hex home for apps with custom logos.
• You can also now enable Dark mode from the command palette:

Bugs squashed​

• We made a bunch of fixes to our dbt integration under the hood.
• Dark mode had some lingering light elements - now appropriately dimmed.

Welcome to another edition of ye olde Hexe Product Updates.

PS: We’re going to the Snowflake Summit in June, and we’d love to see you there! If you’re going too, drop us a line (you can just reply to the email) and we’ll pack something extra special in our bags for you 💜.

🔄 GitHub sync​

We've recently released several features in service of a broader vision to bring software engineering best practices for collaboration and peer review to data work. There’s been one last feature missing.

We’re really excited to show you the final piece of the Hex versioning suite: GitHub sync. Synced Hex projects will store every new version as a commit to a development branch in GitHub; the published app in Hex will remain in sync with the main branch.

As you build, save versions, and publish work, Hex will manage branches for you and automatically push to main when necessary. If you make changes to a project from outside of Hex, it’s easy to pull and sync them back into Hex.

You can also choose to require Pull Requests, which will add a step to the publishing workflow that requires a PR to be approved and merged on GitHub before a new Hex version can be published.

This unlocks a host of powerful abilities, allowing you to programmatically access and edit Hex projects. Some things you might find yourself doing:

• Running a find & replace across all Hex projects to replace a table name that’s been changed.
• Integrating with your project tracking tool to auto-close tasks when a project is published.
• Enforcing some über-custom linting scripts on your Hex code for org-specific code rules.
• Hooking up a printer to automatically spit out a hard copy of every Hex version so you can proudly send them to your family to stick on the fridge.

Read more on the Blog

📝 Text cell​

We built a Text cell! It supports rich text editing with live preview, markdown syntax, and jinja parameterization. Don’t tell Markdown.

Look, we all love Markdown. Nobody loves Markdown more than me! I, for one, have never had even fleeting thoughts about a different text editing experience. Even when I handwrite letters to my pen pals, I make sure to write _ and ** for emphasis.

But I mean, sometimes it could be kind of nice to format a block of text without having to wrap it in special characters... Not usually, of course, but I suppose just between you and me it sounds a little dreamy to select some text and have a little toolbar that could change the formatting right there. But that’s probably crazy. Markdown! **It’s the best**.

So I know you all probably barely even want this, but we wanted to put it out there just to, you know, see if it's cool, almost as a joke. ha ha! rich text!? never even heard of it.

🏴‍☠️ Custom logo​

Admins can now upload a custom logo that will be displayed on published apps. Other parts of Hex, like the home page and Notebook view, will continue to use the default Hex logo.

We hope this helps make your published apps really pop and feel like key pieces of your organization’s knowledge.

Other improvements​

• Columns in the schema browser are now ordered by how they appear in your table, not alphabetical order. For existing tables, you’ll need to kick off a schema refresh to see this.
• The dbt integration now works with Databricks connections.

Bugs squashed​

• Columns of type TINYINT UNSIGNED were causing SQL cells to error, but are now supported.
• We’ve resolved problems that were causing some projects to be very slow to publish.
• We found and fixed a sneaky race condition that occasionally led to blank input parameters in published apps.
• Duplicating a Single value cell now appropriately duplicates all formatting options.
• We fixed an issue that caused multi-select inputs to sometimes display cached values that were no longer valid.

This round of Hex updates includes some long-requested functionality, significant improvements to core Hex workflows that you use every day... and some dark, unspeakable magic. The usual!

🐍 Dark mode​

I was in the library the other day, in the Restricted Section, and I read something rather odd about a bit of rare product functionality. It’s called, as I understand it... Dark Mode.

You can enable Dark Mode from the new User Preferences section of the Settings, where Theme options reside. You can sync Hex to your system mode, or choose to always be Light/Dark.

Check out a demo here

Every single screen of Hex has been carefully themed, from dialogs to inputs and even charts. We'll be adding more themes in the near future, like a dimmed theme and colorblind friendly mode.

👀 Project view stats​

Ever wonder if anyone’s actually looking at that app you built? We talk a lot about “turning data into knowledge” and making sure that your data work has impact— one of the simplest ways to do that is to just see if anyone’s picking up what you’re putting down.

Check out the Published App Views tracker in the bottom left of the sidebar, next to the memory indicator. You can click “See more” to get a detailed access log.

⚙️ User preferences (and Vim mode)​

We’ve already covered most of the contents of the User preferences page, but expect more configuration options to become available here as we build. In addition to Theme settings, Vim key bindings are now available for Code, SQL, and Markdown cells.

🔔 Slack notifications​

I only check my email every two weeks, on Thursdays when I send these release notes. Outside of that, it’s all Slack. This means I regularly miss important Hex notifications, like project access requests, comments, and new shared apps.

You can now choose to receive all or some notifications via Slack, in addition to (or instead of!) email. Head to your Settings > Notification preferences to authorize Slack and configure notification settings.

🗑️ Move to Trash​

Hex now “soft deletes” projects rather than permanently (and slowly!) deleting them. You’ll select “Move to trash” instead of “Delete”, and have access to restore projects from the Trash for 7 days before permanent deletion.

🥧 Pie charts​

We once swore a vow to never add Pie charts to Hex, but we’ve listened to your feedback, put the user first, and... though it pains us to even say it...

released Pie Charts 🥧.

We’re calling them “Arc” charts though, which is both technically correct and protects our Unbreakable Vow.

🖍️ Notebook diff​

When publishing, in addition to the app preview you’ll now see a "Notebook diff" tab that tracks changes to outputs, code, and configuration on a cell-by-cell basis.

This means changes to backend logic are readily visible when publishing new app versions, even for cells that aren’t directly displayed in the app layout.

New cells, updated outputs, markdown grammar fixes, and tweaked SQL queries are all visible and easily auditable before publishing.

💅 Chart improvements​

We’re continuing to add polish to Chart cells with a couple of helpful improvements to standard chart functionality.

Area Charts are now supported, and pair especially well with Line layers when visualizing data that’s lightly related.

You can sort chart axes by the values of the other axis. This sounds weird no matter how you say it for some reason, so here’s a nice GIF example! PS: this only works for ordinal and nominal axis types.

Duplicate chart as Python cell now outputs a Python dict, not a JSON string. This makes custom chart configs a lot more readable, and more easily parameterized based on app logic.

Other improvements​

• You’ll notice lots of visual polish around the app, from new icons to pixel perfect alignment.
• We’ve added a “Contact an admin” option to several places where previously, we just informed users that only admins could take a certain action.

Bugs squashed​

• Some organizations were missing the default set of statuses and categories. We fixed this, and repopulated your empty lists. If you’ve suddenly had this feature appear, welcome!

• If you were using $dollar sign math syntax$ in Markdown cells, you probably noticed some strange formatting behavior. We’ve fixed this, but now you have to use $$double dollar sign syntax$$ to achieve the same effect— just like Notion!

• Multi-select parameters were accidentally able to reference their own output as input, which was fun but not that useful.