Components

The platform is divided into various components to keep it modular, to enable easy scaling for cloud-based deployments, and to manage high loads without much change. Each component can be installed on a separate machine, or any subset can be installed on the same machine.

The components are:

Component	Purpose
Web Application Server	The web application server for the analytical UI of the platform
API - Celery worker	The worker for asynchronous API tasks
Spark - Celery worker	The worker for asynchronous Spark tasks
Jupyter Notebook	The Jupyter Notebook server for free-form analytical use
File Management	The file management server to manage files
Metadata Database (SQL RDBMS)	The database with all metadata provided in the Web Application
Messaging Queue (Redis)	The messaging queue to orchestrate worker tasks
Authentication Provider	The identity and auth provider for access and permissions
Proxy / Load Balancers	Load balancers and proxies to simplify the install

For how these components communicate, see Networking. For guidance on which ones to scale, see Scalability and Sizing.