All the features shipping today are highly requested— I’ve personally fielded many requests from you all for some of these, so I hope you enjoy! The Data Manager is a particularly impactful new feature, but I personally am irrationally excited for the CSV UPLOAD CELL!!!1111
📖 Data Manager public beta
Today we’re opening public beta access to the Data Manager, a metadata tool that’s purpose-built to enhance Hex Magic’s SQL writing skills and custom tune them to your organization’s data.
The Data Manager provides a nice, simple interface for you to access and edit four different kinds of metadata about your data connections:
- Schema, table, and column information from your data warehouse.
- All your dbt documentation, live synced from dbt Cloud or manually added from dbt Core.
- Priority rankings that determine which schemas, tables, and columns should be included, excluded, or prioritized for Magic AI.
- Any additional descriptions or custom metadata you want to provide.
Hex uses this metadata to create vector embeddings to power the semantic search algorithms that are the secret sauce in Magic’s SQL generation engine. Better metadata = better embeddings, and better embeddings = better SQL generation.
The coolest part is that you can provide really custom guidance using normal natural language. Magic is incredibly responsive to things like “only use this table if the prompt explicitly requires raw stripe data, otherwise use fct_orders”, or “this column contains a stringified array of items”. You can essentially educate it as you would a human member of your data team!
I spent about 10 minutes setting up the Data Manager on an internal connection yesterday, and was able to improve SQL generation success rates from 82% to a dazzling 96% on our prompt evaluation suite!
We just published a very detailed blog that introduces the Data Manager and highlights some real examples of using it to tune SQL generation, with specific advice on how to leverage it yourself.
Data Manager is available for workspaces on the Professional plan and above. Workspace admins can find the Data Manager from the bottom left of the homepage sidebar, or at https://app.hex.tech/your-workspace-id/data.
🆙 CSV upload input
The CSV upload input parameter… lets app users upload CSV files! Need I say more?
It also automatically parses the uploaded file and returns a DataFrame for use in downstream cells 🙂.
I’m willfully choosing to not look at the data and am just going off of ✨vibes ✨ here, but I think this has to be one of, if not the most requested feature in our backlog.
And I can see why! Letting end users of apps upload their own data lets you do some pretty cool stuff. A neat hack I threw together: If you pair it with a table input, you can even give app users a full spreadsheet editing experience that culminates in visualization, or data transformation / writeback.
Learn more in the docs.
💼 Presentation mode
Pop your app into Presentation mode to get a full screen experience that splits tabbed apps into individual slides that remain fully interactive.
For years now, we’ve run our regular metrics reviews and Board of Directors meetings out of Hex apps, rather than slide decks. Scrolling through an app always worked just okay. Tabs were a big improvement, but for a real deal meeting, you can’t beat a good old fashioned slide deck.
Unless, of course, you turn an interactive data app into a slide deck, while maintaining the ability to interact with and explore data live, right from the slides 😉. Next time you’re presenting complex findings or doing a metrics review, split your app into tabs and give presentation mode a spin!
Presentation mode is available for workspaces on the Professional plan and above.
🧟 Auto Archive
Workspace admins can now configure auto-archiving of projects based on time elapsed since they were last edited or viewed.
This is a pretty straightforward addition, and we hope it will help keep your workspace tidy. What happens when projects get archived?
- They are excluded from search.
- Any associated scheduled runs are disabled.
- They are NOT deleted, and can be unarchived if needed.
This is a way to gain some ground back from all the Untitled projects and half-done explorations lying around your workspace, as well as old masterpieces that were useful at one point, but aren’t in active service anymore.
Auto archive is available for workspaces on the Team plan and above. Find the config in Settings > Organization.
Other improvements
- Users that have the Slack integration installed can now respond to access requests to projects from within Slack! And if a coworker gets to the request before you do, the Slack message will be updated in-line to let you know. (Note: for now, this only works for users on our multitenant stack, i.e. if your Hex URL starts with app.hex.tech.)
- We’ve improved how chart cells are displayed in the outline sidebar! They are no longer (quite as) squished.
- Single value visualizations now support standard Python negative indexing, so you can select the
-1th row to reliably grab the last row in a dataframe of dynamic length. - Assorted improvements and bug fixes to in app notifications.