MINEO 2023.10: Custom Worker Images

We’re excited to announce the introduction of Custom Worker Images in MINEO version 2023.10. This update streamlines custom package management, promoting reproducibility and greater efficiency in your notebooks and pipelines.

Key benefits:

• Simplified dependencies: Make managing dependencies and reproducibility easier
• Custom packages: Add custom packages with ease
• Performance boost: Enable more responsive notebooks with prebuilt Docker images
• Reproducible environments: Ensure consistent data application environments

Here’s a step-by-step walkthrough of how you can create a custom worker image on MINEO:

Step 1: Create or Import Dockerfile

Kickstart the process by either creating a new worker image or importing an existing Dockerfile into MINEO.

How to import: Simply drag and drop the Dockerfile over the file explorer.

Step 2: Write Your Dockerfile

Commence with a public Docker image containing Python (versions 3.8 to 3.11), or start with a predefined MINEO base image.

Dockerfile guidelines:

• Base image: Use Python 3.8-3.11 or MINEO base image
• Project requirements: Write according to your project specifications
• File system access: Use files stored in MINEO filesystem (e.g., pip install -r requirements.txt)
• Custom dependencies: Include any specific packages your project needs

Step 3: Build the Image

With the Dockerfile ready, initiate the build process directly on MINEO.

Build process:

• Start build: Press the build button to begin
• Monitor progress: View real-time build logs
• Build time: Duration depends on package complexity
• Success indicator: Build completion creates an automatic tag
• Project availability: Tag becomes available for immediate use

Step 4: Assign the Built Image

Once you have a tag, assign it to your desired resources.

Assignment options:

• Whole project: Apply to entire project scope
• Specific notebooks: Target individual notebooks
• Pipeline selection: Choose specific pipelines
• Flexible deployment: Mix and match as needed

Our commitment to improving the user experience and delivering reliable data environments is reflected in this upgrade. The inclusion of custom worker images makes the MINEO platform an even stronger resource for creating, sharing, and deploying data applications.

Explore the improved flexibility and streamlined workflow of MINEO version 2023.10 by utilizing Custom Worker Images, and take your data apps development to a higher level.

Happy coding!