Changelog

Today, we’re excited to share initial results from some big performance initiatives we've been working on, including 90% reductions in frontend page load time for large projects, up to 10x improvements to execution speeds for projects without Python code, and more!

These are meaningful improvements, but we know there’s a lot of work still to be done. We deeply understand this and have entire teams internally that continue to be dedicated to making Hex faster 🫡. 

😴 Lazy Dataframes​

Certain types of projects now execute up to 5-10x faster by skipping unnecessary pandas DataFrame creation.

We’ve rolled out a complete overhaul of the fundamental data interchange format cells in Hex use to store and manipulate data.

Instead of materializing slow, inefficient pandas DataFrames into memory to power our no-code cells, our new lazy loading architecture stores data as Arrow files in S3 and provides a lightweight “Lazy Dataframe” stub for cells to reference and pass around. When cells need to run operations on a dataset, they issue SQL queries directly against the remote arrow files using DuckDB.

This means data is never sent to the Python kernel unless it’s required by a Python cell. No slow, unnecessary dataframes are created, and most importantly: everything still works exactly the same way with no noticeable changes for users.

Well, other than speed! This new architecture has massive performance benefits, specifically for large projects that operate on large datasets without Python. Across three test projects, we saw average runtimes go from:

• Specific performance test project: 12s → 1s
• Real internal project: 15-20s → 3-5s
• A critical customer report: >2min → 30s

The under-the-hood details are pretty neat here! Read all about them on the blog: Optimizing Multi-Modal Analysis by Lazy Loading Dataframes

🫥 Frontend Virtualization Improvements​

Big projects (100+ cells) load and render up to 90% faster.

This video shows a project with 1408 cells (!) scrolling quickly and behaving normally. This wouldn't have been possible with our previous system!

We’ve made significant updates to the virtualization engine that powers large projects, to keep the UI snappy and quick even when projects get up to hundreds of cells.

Testing against a particularly large Hex project, our updated virtualization method resulted in:

• 90% reduction in initial notebook render time
• 33% reduction in UI interaction lag
• 14% reduction in render lag during scrolling
• 10% reduction in overall memory usage

🍿 Fast Kernel Startup​

tl;dr: Fresh project runs are up to 10s quicker, thanks to major optimizations on kernel startup time.

We’ve made many improvements to the way we initialize Python kernels for projects, bringing p95 kernel startup overhead from 12s down towards 1s.

Many of these fixes were little optimizations on already fairly quick workloads. For example, we reduced the time needed to locate a particular kernel in our network from 200ms → 10ms. We also found some big wins though, saving up to 8s on every kernel initialization by optimizing the way we spun up their initial code.

We’ll keep looking for improvements big and small here!

🧭 Faster Explores​

Exploring from charts is 5-10x quicker, unburned by the weight of ...the ancestors?

When you Explore off a chart, we used to obtain and execute all of the “ancestor cells” of that chart in order to create the data needed to power the Explore. This could make exploring really slow, depending on how a particular project was structured.

We can now instantly fetch just the immediate dataset required to power an explore, rather than derive it on the fly from all the ancestor cells.

The impact of this is variable depending on how slow or numerous those upstream cells are, but it can be quite significant. On a real internal project I used for testing, my explore load time went from 30+ seconds to around 5 seconds.

🏎️ Advanced Compute Profiles (beta)​

We’ve done a lot of work to make our code run faster, but unfortunately we can’t make your code run faster. Oh wait, no, we totally can— if you’re willing to pay for it!

We have two new heavy duty compute profiles available in closed beta, with usage-based billing:

2XL: 8 CPUs, 64 GB memory 4XL: 16 CPUs, 128 GB memory

These larger profiles will let you work with larger datasets and parallelize heavy workloads. We will be experimenting with other profiles in the future, including GPUs.

If you’re interested in joining the beta, please fill out this form. And if you are a Hex user, but still have to run a workload in another environment because of compute requirements, please let us know! We want to meet all your compute needs.

📊 Run Stats (beta)​

Detailed execution data is now available for projects, so you can easily identify bottlenecks and slow queries.

You’ll find Run Stats in the Run dropdown menu, where you’d go to stop or restart a kernel. It gives you detailed execution timing for every cell and the entire project, as well as some of the internal things I’ve mentioned above like kernel initialization.

This should make it a lot easier to pluck out slow queries, long Python blocks, or anything else that seems out of sorts.

Pro-tip: Combine this with the graph viewer to get a really detailed sense of a project’s execution flow and performance!

🚀 Onwards​

We’re really excited about the headway we’ve made so far, but our quest is not even close to over. Perf work is like whack a mole, or maybe an ultramarathon. Perhaps one of those bootcamps where you have to crawl in the mud under barbed wire while someone hollers at you.

Either way, we’re committed to keep improving Hex’s performance on all fronts, from tiny kernel initialization gains to major execution overhauls and frontend rendering improvements.

Please continue to report places where things feel sluggish, because it’s our best data for making improvements. All the things you just read about are directly based off of example projects sent in by users! 

Until next time 🖖.

🧮 Parameterized Calculations​

Input parameters can now be referenced in calculations using standard {{input_name}} syntax.

We built Calculations to let business users dive deeper into data and answer their own “what if” questions without constantly tapping the data team on the shoulder. Now those same users can write calculations that respond to user input, building entire interactive reports end-to-end without writing a single line of SQL or Python.

Here’s a short video where I build a what-if scenario modeling tool. I rely on parameterized calcs and don’t write a single line of SQL or Python code— and the end product is linked here for you to explore.

Building these kinds of custom reports has traditionally required data team support, and now it’s well within the reach of any data-curious explorer.

Learn more in the docs.

🔢 Single Value Aggregations​

Single Value cells can now calculate totals, averages, and other column aggregations.

Can you imagine a world so cold, so savage, so brutal that you’d need to use an intermediate Python cell to sum a column for single value display?

Let that world be nothing but a forgotten memory, an ancient myth.

Just like parameterized calculations, this removes one of the last bits of friction preventing no-code users from building rich, custom apps in Hex. It’s now possible to go from the data browser, to a table display, to a single value cell showing a column’s average without writing any code.

You can even turn any total from a Table Display into a standalone Single Value with just a click:

As part of this upgrade, Single Value cells also accept Query objects in addition to Dataframes, which lets these aggregations run on datasets of any size without pulling results into memory.

🐍 AI Autocomplete for Python​

AI powered code completion is now in beta for Python.

Hex gives you fast, relevant AI code suggestions as you type, based on context from your entire notebook— not just the cell you’re working in. All you have to do to accept is hit Tab.

This feature is in private beta as we iterate on accuracy and the completion UX. Reach out to [email protected] if you’d like to be added to the beta!

🧙 Magic UX updates​

You can now test run AI edits before choosing to accept code or keep iterating.

This tightens the feedback loop of working with Magic and lets you edit with more confidence. Rather than have to accept AI edited code in order to evaluate whether or not it actually does what you want (and then potentially revert or edit it after), you can run the response in a sandbox, and decide whether to keep it or continue iterating.

Magic will also let you know when it can’t help with something. Some functionality isn’t supported by Magic yet, like writeback cells and input parameters, and now Magic can politely let you know when something’s beyond it’s current capabilities.

💬 Hex in the Slack App Directory​

Hex’s Slack integration is listed in the official Slack App Directory, which unblocks access for some workspaces due to security policies.

We heard from some customers that their infosec rules prevented installing Slack integrations that weren’t in the official Slack App Directory. Now those customers can use the Hex integration!

Learn more in the docs or get right to installing.

Other improvements​

• Date diff functions have arrived in Calculations 🎉. You’ll find DiffDays, DiffHours, Diff…. you get the picture. Enjoy!
• Charts will now attempt to cast columns automatically when needed. For example, if you choose a string column for a y-axis that has numbers represented as strings (“1”, “2”, “3”, etc.), Hex will cast them to numbers and show a continuous axis, not a categorical one.
• When picking columns from a multi-select, you will no longer be auto-scrolled back to the top of the list.
• “Copy link with inputs” now includes filter selections as well as input parameters.
• When removing a data connection within a project, you now have the option to replace all instances of it with another data connection.
• There are new filters on the Projects page, including a filter for whether projects require a review, whether they are scheduled, and which collections a project is currently in.
• If your workspace uses auto-archive, we now exclude any projects that have been manually restored from future auto-archiving, so you don’t have to keep whacking moles. Or un-whacking moles? Whatever. You don’t have to worry about moles at all, for any reason.

This month we’ve got some really useful new tools for keeping your Hex workspaces organized, and two simple but impactful updates to AI workflows.

We also published a blog post about these new organizational tools with a lot more in-depth example videos and details. Check it out!

🌪️ Find and organize your projects​

We’ve added more filtering options to project lists, and you can now select and take action on multiple projects at once.

Above, I'm using filters to identify some loose demo apps that were shared publicly, and then quickly adding them to a collection with bulk actions.

From any project list, use checkboxes or shift+click to select multiple projects. Once you've made a selection, you can add to collections, change statuses, add categories, and archive or delete in bulk.

You can also now filter project lists by title, status, category, ownership, and permissions — and sort by all of these, and more.

Here, I'm combining filters to find obsolete duplicates of projects that were meant to be shared externally, but never made it out of the exploratory phase:

Read the blog post for a look at all the new features and some specific recommendations on how to use them to tidy up!

🤝 Transferable project ownership​

Ownership of projects can now be changed over time as teams and responsibilities change, to prevent permissions confusion.

Just like bulk actions and filters, this is a tool to keep your workspace tidy. As teams change, sometimes you need to change ownership of important projects. Now you can make those transfers with just a couple clicks (and it can be done in bulk, too!).

I assume my keen-eyed data audience will immediately realize that this introduces some new metadata... Indeed, every project now has both a Creator and an Owner. Creators are immutable, but Owners are designed to change over time. Both are of course available for filtering and sorting!

Learn more in the docs.

🤖 Smarter, faster AI edits​

Magic AI edits now make line-by-line changes to SQL and Python instead of rewriting everything.

You can also continue editing without accepting or rejecting, and edit conversationally with improved message history.

There's two changes on display in that gif: diff-based editing and continuous editing.

Diff-based editing means Hex returns only the specific line edits necessary to achieve your request, making it faster to work with large queries and complex code using AI. Now, no matter how big your query or code gets, AI edits will return lightning fast.

Continuous editing lets you keep editing without accepting/rejecting the most recent AI response. Combined with a new context memory of your requests, this means you can more naturally iterate on changes with Magic.

I use this all the time now, most commonly to make an edit to one part of some code and then say “awesome, now do that for all the remaining functions”.

The fact you can say “do that” and it knows what "that" means from the prior context seems trivial, but is an absolute game changer for efficiency and flow.

🛡️ Workspace API Tokens​

Admins can now configure workspace API tokens that do not expire.

Hex has always had personal API tokens, with an expiration policy to ensure they're rotated frequently. We’ve heard lots of requests for permanent API tokens. This makes total sense: you don’t want your orchestration flow breaking every 14 days when a token expires.

So we’ve added the ability for admins to set up special workspace tokens that don’t expire! Unlike personal tokens, these can be centrally managed and revoked by any workspace admin, so you don’t lose track of them. We do still recommend you periodically rotate workspace tokens!

Learn more in the docs.

Other improvements​

• Schema Refresh Control: Admins can now restrict who is allowed to manually refresh connection schemas in the data browser. This change, especially combined with our existing refresh-on-a-schedule feature, is a nice way for workspaces to regulate when a schema gets refreshed, and by who!
• We fixed an issue with IsNull() previews in Calculations.
• We are knee deep in some frontend performance work to make big notebooks faster, which you'll hear about later in the summer.

Today, we're taking the powerful data tools we've built for technical practitioners, and opening them up to everyone!

We're releasing end-to-end no-code workflows with spreadsheet-style calculations, data browsing, and updated governance features.

These new workflows live right alongside our existing SQL, Python, and AI tools, so everyone— regardless of technical experience— can come together in Hex to ask and answer questions with data.

🧮 No-code Data Browser​

Our new no-code Data Browser makes it easy for anyone at your company to search for and preview data.

You can search for tables, columns, or descriptions, and then see previews and additional metadata – including some imported from your warehouse or dbt Docs.

Then when you find a table you want to work with, no-code cells can now load data directly from underlying databases, without an intermediate SQL query. Just point at a table, and go. All the bells and whistles work in this context too too, from column formatting to totals and filters.

∑ Calculations​

Hex speaks spreadsheet! We’re adding support for the world’s most popular data language, and introducing spreadsheet-style formulas, right in Hex.

Spreadsheet calcs are available almost anywhere you see a table. Just click the + and boom – you’re back in banking! As of today Hex supports over 60 of the most common spreadsheet functions, with more (including aggregate functions and input parameters!) coming soon.

When you execute a calc, Hex’s compute engine compiles it to dialect-specific SQL, and pushes it down to the full scale of the underlying data. But to make everything feel snappy, Hex runs fast previews, which run on a sample of the data in your browser:

🔏 Endorsed Statuses​

Endorsed statuses provide a visual indication of which projects and data are trusted, governed, and canonical.

Only Admins and Workspace Managers can add an Endorsed Status to a project, providing a layer of trust and governance around your workspace.

They’re also fully customizable, so you can add whatever name, color, and icon best suits your workflow.

Endorsements work great with our new Reviews feature, too: you can set a given status to require a review, so you can be sure you always get an second set of eyes on changes to important projects.

Learn more in the docs.

🔮 Faster, more functional Magic AI tools​

We’ve made a ton of improvements to our AI tools to make them faster and more accurate.

We're releasing:

10x speed improvements for Magic Charts! We changed how we do prompting and parsing for text-to-viz so they generate much faster.

Diff-based editing for Python: now instead of waiting for the whole thing to stream back, we just get the diff back. Much faster!

And an overall ~2x speed boost with the latest GPT4o model.

A note about bringing everyone together with data​

People don't talk as much about the "Modern Data Stack" anymore, but it's amazing to stop and think about how much it revolutionized the backend infrastructure of data analytics, making fancy software engineering best practices and powerful tools accessible to everyone.

The next thought I always have is just how fragmented, messy, and scrappy the frontend of data analytics still is. SO many things are way harder than they need to be!

At Hex, we believe that there's a better future just around the corner in the form of multi-modal workspaces like Hex— unified platforms that let you work seamlessly between code, no-code, and natural language prompts to build with data alongside anyone from your company.

With today's updates (and everything else on the horizon 😉) we're building towards that future as fast as we can.

And if you made it this far... what are you waiting for! Go write some calcs, browse for some data, and consume some delicious, trusted, certified projects. mmmm, certified projects.

Get ready for next month's official Spring Launch on May 29— we have some amazing things coming down the line, so stay tuned! The week after that, I’ll be hosting a hands-on demo of all the new features that you can sign up for now.

🕵️‍♂️ Audit logs​

There is no Pepe Silvia. The man does not exist, okay?
— Charlie Kelly

Enterprise customers can now access a complete event log of their Hex workspace, detailing every breadcrumb from user role edits to project and data access.

If you’re interested in reviewing this kind of data, chances are you’ve already got a conspiracy wall SIEM tool to centralize and analyze all your logs. This feature gives you access to raw log exports that we can pipe right into the log analysis tool of your choice!

This feature is available to workspaces on our Enterprise plan. Please reach out to your Hex point of contact for more information.

📤 Expanded Git support​

Git project export and package import now supports Bitbucket and GitHub Enterprise. Components can now be exported to Git.

We’ve also improved a lot of behind-the-scenes action here to avoid sticky Git situations and generally streamline export flows. Git export already worked for GitLab and GitHub, and that hasn’t changed.

Why use Git export? Now that Reviews provide a first-class approval workflow entirely inside Hex (check them out if you haven’t yet!), Git export is primarily useful for maintaining an audit log of changes to projects. If you enable it on a project, you’ll be able to see line-by-line diffs of every saved and published version of that project as commits in the repo of your choice.

Pro tip: Combining Git export with workspace audit logs gives enterprise customers a detailed audit record of every change made in a workspace, from access and configuration to actual code changes.

Git export is available to workspaces on the Team plan or higher. GitHub Enterprise is only available to workspaces on the Enterprise plan.

🗃️ ListProjects API endpoint​

You can now list projects from the API.

This endpoint lists projects. But wait! It also returns a ton of project metadata like status and categories, so you can easily queue up custom runs of multiple projects based on metadata filters, without having to make a bunch of calls to GetProject.

Learn more in the interactive API documentation.

This endpoint is in beta. We’d love to hear your feedback and how we can make it (and the entire Hex API) more useful for your project execution needs.

💫 Magic chart performance​

Magic now generates charts up to 10x faster. This clip is not sped up or cached:

Magic lets you use AI to generate queries, code, and visualizations faster than you ever could by hand.

Except… the visualizations weren’t always faster than you could do by hand. It depends how fast you click, but we clocked new chart generations at ~10 seconds in our tests— way too slow.

Now, AI chart generation should only around 1 second! Right now this boost is only for new charts, but we’re working on bringing this to edits too.

Pro tip: When you use Magic in Generate mode, it automatically decides whether or not you want a chart based on your prompt. Try saying "build me a chart of x" or "query data about users and then visualize it as a 100% column chart"

Other improvements​

• NaNs and nulls are now visually quieter than other values in tables, making them easier to skim.
• We fixed issues with browser performance when resizing a Hex window containing many charts.
• We’ve upgraded our emoji support to unicode 15, so you can use the 🫎, 🫠, and other important new emojis in comments and text cells.
• Comments also now support rich text, like code blocks and bold font.
• We fixed a bug that could prevent projects being removed from collections.