Improving Hex Magic
The best way to think about Magic is similar to the way you’d think about enabling a new college graduate - they have all the core fundamentals and know how to write great SQL or Python but they lack business context. Being faced with an abundance of new information can obfuscate their understanding and lead to incorrect assumptions.
A great proxy for “how to make Magic better” is the same stuff that would be great to enable this same persona. The ideal prompting should also have similar context by providing clear and specific direction as if you’re talking to someone who knows how to code but doesn’t have any business context. Great data curation setup is the #1 lever you can pull to augment Magic prompts and help return higher quality results.
This tutorial outlines steps that help provide Magic a focused view on relevant information, leading to more accurate suggestions.
Hex Magic is meant as a way to augment—not replace—human insight and judgement. You should not rely on AI-generated code to be accurate, complete, or free from bias.
Data Connection Setup
The first thing to consider is if your data connections need curation. Often a small subject of teams in Hex need access to the full warehouse, but the majority of users need only a subset of databases and schemas. Creating additional data connections with curated data access has benefits:
- You can tailor that data connection to only have access to the schemas/databases that the majority of users need to use, thereby giving a focused view of the data.
- You can make that data connection the default data connection, directing new users in Hex to use this connection to start.
You can make this data connection and either:
- Create a new role/permissioning structure in your warehouse directly. This will prevent users from querying tables outside of the schema/databases this role has access to.
- Create a duplicate data connection but use Hex's schema filter feature - this will only pull in schema details based on regex rules you set. This is a lightweight way to curate your data connection without any extra work in the data warehouse.
Schema filtering is not a security feature and users can still query the underlying tables directly if the user has proper permissions to do so.
Once you have your connection set up, you can set up scheduled refreshes to ensure we’re regularly resyncing and making sure Magic has access to up-to-date schema information and metadata.
Giving Magic the right context
For more examples, check out our Managing Magic blog post.
Endorsements
Adding an endorsed status to a database, schema or table is the easiest and fastest way to help Magic (and end users!) know what data is “Approved” or “Trusted” by data leaders. Endorsed data will be prioritized for Magic usage. Child data objects will inherit their parents Endorsed status and also will be prioritized - so it's easy to endorse an entire database or schema with just one click!
Magic Exclusions
If you want to maintain access in the Data browser to certain databases/tables/schemas but never want Magic to use these tables, you can toggle the “Include/Exclude for Magic” setting. Child data objects will inherit their parents exclusion state, so if you've excluded an entire database, all schemas, tables and columns contained within would also be excluded from Magic.
Adding Metadata
Table and column descriptions saved in your warehouse get synced with every schema refresh. When available, Magic can use this context to generate better suggestions. If you use dbt, you can take advantage of our dbt Cloud integration - once set up, Hex will sync all your descriptions which Magic will automatically ingest.
What kind of metadata is helpful for Magic?
Below are some guidelines to consider when thinking about what data to include to help Magic:
- For a table:
- Information about what a table should be used for and what things can be calculated with it
- For a column:
- Add enumerated values for all low cardinality columns
- For text values, add an example in the description and describe a pattern it might follow:
- Example:
State: Text column following the pattern: CA , AZ , NV
- Example:
- Other considerations:
- Jargon - Companies often have terms or phrases that are very specific to them and LLM’s might not be familiar. Try and explain what specific words or phrases mean to give Magic the best chance of understanding this terminology.
- Example:
The term “S0” is used to refer to a pipeline opportunity that's in stage 0
- Example:
- Synonyms(‼️) - Companies sometimes use multiple words to describe the same concept. Giving this context to Magic can help provide suggestions more consistently when folks describe things with different verbiage.
- Example:
The words “organization” and “workspace” are used interchangeably, but “organization” is how it's represented across all data tables
- Example:
- Jargon - Companies often have terms or phrases that are very specific to them and LLM’s might not be familiar. Try and explain what specific words or phrases mean to give Magic the best chance of understanding this terminology.
Above is an excerpt from our data team's Coalesce talk on documentation best practices - you can listen in here!
How do I know if my descriptions are helpful?
If you want to prototype descriptions and see how Magic does with them, you can directly edit the description fields in Hex within the Data browser UI. You can then ask Magic your question and observe the results, updating the descriptions as needed!
If you want Magic to use a specific table, you can @mention the table in the prompt bar.
Our suggestion is to prioritize the top 10 tables you expect folks to be using and filter out or exclude STAGING/DEV/RAW schemas. If you prioritize endorsing and documenting these 10 tables, you should see a large increase in the accuracy and perfomance of Magic!