Databricks Architecture ¶
Databricks operates out of a control plane and a compute plane.
-
The
control plane
includes the backend services that Databricks manages in your Databricks account. Notebook commands and many other workspace configurations are stored in the control plane and encrypted at rest. -
The
compute plane
is where your data is processed.-
For most Databricks computation, the compute resources are in your AWS account in what is called the
classic compute plane
. This refers to the network in your AWS account and its resources. Databricks uses the classic compute plane for your notebooks, jobs, and for pro and classic Databricks SQL warehouses. -
For
serverless SQL warehouses
orModel Serving
, the serverless compute resources run in a serverless compute plane in your Databricks account.
-
Important
Previously, Databricks referred to the compute plane
as the data plane
Serverless Architecture¶
Azure Databricks
creates a serverless compute plane
in the same Azure region as your workspace’s classic compute plane
.
Arch for model serving is shown below