Scripts for provisioning data science tools
-
Updated
May 26, 2018 - Shell
Scripts for provisioning data science tools
Projeto do Curso "Criando um Ecossistema Hadoop Totalmente Gerenciado com Google Cloud Dataproc" do Bootcamp Data Engineer da Digital Innovation One
🐳 Docker container for Spark on college (HHS).
GCP Dataproc mapreduce sample with PySpark
Installation instructions for pyspark and a kernel with jupyter
Serverless PySpark
The Data Pipeline using Google Cloud Dataproc, Cloud Storage and BigQuery
Building a Spark standalone cluster with Docker
This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.
Setting up Data Pipeline in AWS using AWS Data Pipeline, S3 and EMR
ezpz pyspark dev environment with docker
Creating gcloud dataproc cluster with this github action
Scalable Spark Docker image that can works on Docker Compose and Kubernetes
P.O.C Spark On Kubernetes
Local integration test setup for pyspark with AWS through Localstack
Add a description, image, and links to the pyspark topic page so that developers can more easily learn about it.
To associate your repository with the pyspark topic, visit your repo's landing page and select "manage topics."