Create a Dremio Data Source

Known limitations

  • Date arithmetic:
    • Not all period-over-period functionality works due to incorrectly translated TIMESTAMPADD
  • Functions:
    • Some WINDOW frames are not supported
    • GREATEST and LEAST functions do not work correctly, if a second parameter is used
      • Example: GREATEST(column_name, 5000)
    • Rarely, if a report works with empty dimensionality, Dremio incorrectly adds the clause OFFSET 0 ROWS FETCH NEXT 0 ROWS ONLY, which causes the report to return 0 rows.
  • Only UTF-8 characters are supported
  • Rarely, a duplicate column alias may be generated, resulting in the corresponding report failing to execute (internal error)
  • When pre-aggregation caching is enabled the report may fail with an internal error
    • The caching mechanism creates tables for each pre-aggregation and utilizes it in outer aggregations
    • Dremio does not register tables into its catalog, if populating SELECT statements return 0 rows

Prepare Dremio for GoodData

To learn how to register Data Sources to Dremio, refer to the official Dremio documentation for connecting a Data Source. To access the Dremio web console, load localhost:9047 in your web browser. Register the user and password for later use when you create the Data Source definition.

Depending on the Data Source you use, additional preparation may be necessary to integrate your Data Source Manager with GoodData. For general considerations, refer to Preparing Data Source Managers for GoodData.

Data Sources Providing Metadata

If you use a Data Source that accommodates metadata (for example, Postgres), consider the following to enure your scan of the Data Sources returns data:

  • Database tables and views can be scanned only if they have been queried in Dremio.
  • Alternatively, you can create Dremio datasets on top of the tables or views to have them available as views without needing to query Dremio.

Data Sources that do not Provide Metadata

If you use a Data Source that does not accommodate metadata, you must always create the datasets.

Data Source Details

The following considerations apply when you are configuring the JDBC URL:

  • If you start Dremio as docker container, you can connect using this URL: jdbc:dremio:direct=dremio:31010.
  • If you run Dremio outside of a docker container, consult the official Dremio documentation for configuring the JDBC URL.
  • There are almost no limits for the driver setup except insecure parameters like e.g. trustStorePassword. For all possibilities, see the official documentation.
  • Basic authentication is supported. Specify the user and password accordingly.
  • You can set enableCaching to true and cachePath to ["$scratch"]

Performance Tips

If you want to query large datasets or even join large datasets from different data sources, we recommend that you use the Dremio reflections feature.

Query Timeout

Query timeout is not supported for Dremio yet.