posstar.blogg.se - Amazon redshift data warehousing

AMAZON REDSHIFT DATA WAREHOUSING DRIVER
AMAZON REDSHIFT DATA WAREHOUSING SERIES

More than 10,000 companies use Redshift, according to HG Insights. The technology on which Redshift is based was licensed from ParAccel Analytic Database in 2012, which in turn was loosely based on PostgreSQL. It allows you to run complex analytic queries against petabytes of structured data, using sophisticated query optimization, columnar storage on high-performance local disks, and massively parallel query execution.” Amazon Redshift overviewĪs defined by Amazon, “ Amazon Redshift is a fast, fully managed data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing Business Intelligence (BI) tools. They're similar in many ways, but anyone who's comparing cloud data warehouses should consider how their unique features can contribute to an organization's data analytics infrastructure. Google BigQuery: a comparisonĪmazon Redshift and Google BigQuery are the Coke and Pepsi of data warehouses: two comparable fully managed petabyte-scale cloud data warehouses. SONG_OPEN_UI_ELEMENT_CLICKED | ios | 00:00:00.Amazon Redshift vs. Select event, platform, time, jsonb_pretty(data) from events_today limit 5

A TimescaleDB hypertable model would allow us to accommodate the schemaless JSON structure of our events.

It allowed us to deploy it on our self-hosted PostgreSQL data warehouse.

TimescaleDB is an actively developed, open-source solution.

The events’ raw data is available in a PostgreSQL schema alongside all our other business intelligence data.

Our data analysts are well-versed in SQL and PostgreSQL.

We found out about TimescaleDB from being part of the PostgreSQL community, and when we were faced with the problem at hand, it was a natural way forward for us.Īfter doing our research, we realized that TimescaleDB suited our needs perfectly. With time, we realized we needed to scale our previous Amazon Redshift data warehouse model into a more suitable solution for categorical, time-series data analysis. For example, a Learn Session event would have a songId and a learnMode, while a Subscription Offer interaction event would have a productId, etc. The properties field is a schemaless object that depends on the particular event type that we track from the app. Every day, we receive millions of incoming events, tracked around our product, in the format: When we launch a new feature, we typically A/B test it and evaluate its impact based on measuring key performance indicators (KPIs), which are predefined for the test. To learn more, read What Is Time-Series Data (With Examples). We found out about TimescaleDB from being part of the PostgreSQL community”įor example, a Learn Session starts at a given timestamp for a user, and we record this event.Įditor’s Note: Time-series data is a sequence of data points collected over time intervals, allowing us to track changes over time. “We realized we needed to scale our previous Amazon Redshift data warehouse model into a more suitable solution for categorical, time-series data analysis. This data consists of user events, which we track from our product.

AMAZON REDSHIFT DATA WAREHOUSING DRIVER

Our experimentation-a major driver of our growth-is powered by the data analysis of user behavioral data (app usage). We are a business that depends heavily on analytics for business decision-making. Not for me personally, though, but still. Many of us take on more than one role, and for many of us, flowkey has been the first significant career step. We also have Customer Support, Product, Engineering, Data, and Operations teams.

AMAZON REDSHIFT DATA WAREHOUSING SERIES

We have a Marketing team (responsible for user acquisition, customer relationship management, collaborations, etc.), a Creative team (building all of our visual content, our design, and advertising), Course and Song teams (creating our in-app learning content-e.g., the courses series and the piano renditions of the songs in our library). We are a team of around 40 people, with more than 10 of us working in Data and Engineering. Here is a video from our founder, Jonas Gößling, explaining how it all started. The company was launched in 2015 and quickly became one of the global leaders in its category. About the Companyįlowkey is a leading app for learning to play the piano, with over 10 million registered users in more than 100 countries.

In this edition, Nikola Chochkov, lead data scientist at flowkey, shares how his team migrated from Amazon Redshift to TimescaleDB and is driving rapid growth and experimentation by analyzing the users’ behavioral data using TimescaleDB along with Metabase or Jupyter/Rmarkedown Notebooks.

This is an installment of our “Community Member Spotlight” series, where we invite our customers to share their work, shining a light on their success and inspiring others with new ways to use technology to solve problems.