Infrastructure That Thinks Ahead
From scalable clusters to secure containers — discover the tech backbone driving DSRS innovation.
At the core of DSRS lies a powerful, flexible, and secure infrastructure designed to support advanced research and data-driven innovation. Our architecture integrates leading open-source tools, cloud platforms, and virtualized resources to enable scalable, on-demand computing environments for diverse research needs.
The DSRS infrastructure's backbone is a high-performance computing cluster consisting of over 640 cores, 2.5 TB of RAM, and over 45TB of shared storage, in addition to 2.8 TB of SSD storage. This set up provides substantial computational power to support a multitude of research projects simultaneously.
The cluster, managed by OpenStack, is utilized to provide virtual machines for individual applications, including our blockchain applications and other environments tailored to specific research groups. Within this architecture, we also deploy a Kubernetes cluster using Terraform and Rancher, both cloud technologies that offer flexibility and scalability.
On top of this Kubernetes cluster, we deploy various applications managed by Argo CD, a continuous integration platform. This platform helps maintain the stability and reliability of our services, thanks to its version control and preservation of the cluster status after every change. All of our manifests and configurations are on Github which is directly connected and in sync with Argo.
An important part of the infrastructure is our Jupyter Hub environment, used both internally and by other research groups within the college. We have the capability to provide a hub to a group or to deploy individual labs for particular needs with a line of configuration. This environment, complemented by frameworks like Dask, Spark, H2O, and funcX, that supports the parallel execution of machine learning models and makes interactive, data-driven research readily accessible.
To ensure our infrastructure supports a wide variety of research activities, we've incorporated different databases such as MongoDB, PostgreSQL, and MySQL. Moreover, we have a dropbox-like infrastructure for seamless file upload, download, and sharing as well as S3 equivalent storage via MinIO. The security of our platform is a high priority, with Dex handling authentication and authorization.
