This was inspired by a thread in the dbt slack community. A need came up for developers to be able to download a fully working and configured project which they could just run (or modify) to demonstrate an aspect of dbt or share a problem/solution.

This is an approach for sharing CLI/dbt-core projects in a self-contained way. I’ve used SQLite so that regardless of the database a developer may usually use (and have installed, hosted or other access to), minimal extra installation is required to demo an aspect of dbt functionality. In short:

From an empty directory, to dbt run in under 5 minutes.

The project, code and documentation can be found on my GitHub here:

https://github.com/d-roman-halliday/SQLite-DBT-Example

What this isn’t!

Importantly, this isn’t a “getting started with dbt” project (although it uses their example models), and it’s not a guide on any specific dbt approach outside of structuring a project and repository to be self-contained.

What is it for?

This is a suggestion for a common approach to creating an easily distributable demo project which any developer can use/share/modify. Mostly for the purpose of demonstrating dbt code or methods.

• Worksop projects
• Sample transformations/solutions
• Sample macros
• Sample problems

The key point is that rather than just sharing a few example files or just a git repository of code. This approach is about a working dbt project which can be used, modified and then thrown away.

What’s significant about it?

This example (and anything built like it/extending on it) is self-contained, so it can be replaced and deleted with minimal impact on the system it’s being run on. To achieve this, the project uses the dbt-SQLite database connector, with a local (to the project) profiles.yml configuration file.

This can have a running pre-built dbt project without installing any large database software, or modifying existing local configurations. Everything is self-contained in a python virtual environment and local directory structure (checked out from GitHub).

How do I use it and what do I do with it?

View the existing example

Follow the steps under ‘Step by step’ Checkout & Execution to pull and run the project. In short, the below steps (note variations for windows/git access):

# Create a working directory
mkdir dbt_demo
cd dbt_demo

# Clone the code from git (use one of)
git clone git@github.com:d-roman-halliday/SQLite-DBT-Example.git
git clone https://github.com/d-roman-halliday/SQLite-DBT-Example.git

# Create a python virtual environment (the second option shown as a Windows example, because python isn't always configured in the path on windows)
python -m venv venv
"C:\Program Files\Python310\python.exe" -m venv venv

# Start the virtual environment (The second line is for cmd under windows)...
source venv/bin/activate
venv\Scripts\activate.bat

# ...and install `dbt-sqlite`
pip install --upgrade pip
pip install dbt-SQLite

#Run the project (changing into directory, showing both options for run and build (which includes tests))
cd SQLite-DBT-Example
cd demo

dbt run

dbt build

There is also a demonstration script which does much of the installation work automated, this is documented under Setup Script. I recommend caution and reading the script before importing it and executing it as shown in the documentation, as it configures the virtual environment and checks out code from GitHub.

Create a distributable project

I hope this helps others, I’m planning on using it as an approach for some examples in the future.

To use the same approach, people can either

1. Copy the repository as is and modify it
2. Follow the same steps I made under Installation & Configuration Command History:

SQLite Database Browser

The only thing you might need to install (or use a standalone binary for) is a browser for SQLite databases, the dbt-sqlite python library can create and update the SQLite database, but it’s not helpful for browsing (to view the changed data). To view the database you will need either SQLite or an application which can browse the database file such as DB Browser for SQLite.