Skip to content

Releases: awslabs/data-on-eks

v1.2.1

05 May 18:18
a796297

Choose a tag to compare

What's Changed

  • docs: How to retrieve Argo Workflow WebUI url by @berry2012 in #873
  • docs: Adding Spark Best Practices with new guidelines by @arunnalpet in #856
  • feat: Datahub upgrade and security patch by @bbgu1 in #871
  • feat: Add Ray Data Spark logs processing example with Apache Iceberg by @ratnopam in #877
  • docs: Update website doc to link to the blueprint by @ratnopam in #878
  • feat: Spark on EKS Blueprint Update by @vara-bonthu in #880
  • fix: Add shs helm provisioning timeout by @ratnopam in #881
  • feat: Deploying gp3 storage class before the addons to avoud race condition by @vara-bonthu in #883
  • feat: Added bedrock IAM policy to JupyterHub Notebooks by @vara-bonthu in #884
  • fix: Add sonnet perms by @nabuskey in #886
  • fix: Fix tf lint by @nabuskey in #887
  • feat: VPC Module update to create subnets dynamically for AZ = 2 to 4 by @vara-bonthu in #889
  • fix: Add spark history server enable variable by @ratnopam in #882
  • chore: Pin IRSA module version by @alanty in #893
  • docs: Added Spark workshop links to Website homepage by @vara-bonthu in #896
  • feat: Add Example for Apache Beam on Spark on EKS by @bbgu1 in #852
  • fix: Airflow SSH Key Fixes by @jagpk in #898
  • feat: Kafka cluster validation script by @saiteja313 in #850
  • chore(deps): bump tar-fs from 3.0.9 to 3.1.1 in /website by @dependabot[bot] in #902
  • feat: Version upgrade for EMR / Flink and adding autoscaler example by @bbgu1 in #897
  • docs: Update EMR on EKS with Flink Streaming documentation by @Eriall1 in #900
  • refactor: CNPG on EKS bug fixes, documentation updates by @reverse in #899
  • feat: TPCDS Benchmarks for "Spark + Gluten +Velox" Vs Native Spark by @vara-bonthu in #903
  • feat: Security fix and pre-commit fixes by @vara-bonthu in #906
  • fix: Improve install/cleanup dependencies. by @alanty in #901
  • fix: Upstream CSI Driver issues by @meetreks in #910
  • feat: Add celeborn benchmark by @nabuskey in #918
  • chore(deps-dev): bump js-yaml from 4.1.0 to 4.1.1 in /website by @dependabot[bot] in #914
  • chore(deps): bump node-forge from 1.3.1 to 1.3.2 in /website by @dependabot[bot] in #919
  • chore(deps): bump brotli from 1.1.0 to 1.2.0 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #916
  • fix: Update github actions to publish the website by @vara-bonthu in #920
  • fix: Spelling and grammar errors by @vara-bonthu in #921
  • chore(deps): bump werkzeug from 3.1.3 to 3.1.4 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #923
  • chore(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /website by @dependabot[bot] in #922
  • fix: Update celeborn benchmark with more data by @nabuskey in #932
  • chore(deps): bump urllib3 from 2.5.0 to 2.6.3 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #930
  • chore(deps): bump werkzeug from 3.1.4 to 3.1.5 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #931
  • feat: Data on EKS v2 by @nabuskey in #935
  • fix: Update website release build directory by @nabuskey in #940
  • chore(deps): bump urllib3 from 2.5.0 to 2.6.3 in /data-stacks/workshop/src/data-flow by @dependabot[bot] in #939
  • chore(deps): bump pydantic from 2.0.3 to 2.4.0 in /data-stacks/workshop/src/data-flow by @dependabot[bot] in #938
  • chore(deps): bump pyasn1 from 0.6.1 to 0.6.2 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #937
  • chore(deps): bump werkzeug from 3.1.3 to 3.1.5 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #936
  • chore(deps): bump pyasn1 from 0.6.1 to 0.6.2 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #933
  • chore(deps): bump apache-airflow from 3.1.0 to 3.1.6 in /data-stacks/airflow-on-eks/examples/dag-bundles by @dependabot[bot] in #944
  • chore(deps): bump brotli from 1.1.0 to 1.2.0 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #942
  • chore(deps): bump urllib3 from 2.5.0 to 2.6.3 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #941
  • fix: Update python packages by @nabuskey in #948
  • chore(deps): bump urllib3 from 2.5.0 to 2.6.3 in /data-stacks/airflow-on-eks/examples/dag-bundles by @dependabot[bot] in #946
  • fix: Fix datahub link by @nabuskey in #951
  • chore(deps): bump requests from 2.32.3 to 2.32.4 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #943
  • chore(deps): bump aiohttp from 3.13.1 to 3.13.3 in /data-stacks/airflow-on-eks/examples/dag-bundles by @dependabot[bot] in #947
  • chore(deps): bump starlette from 0.48.0 to 0.49.1 in /data-stacks/airflow-on-eks/examples/dag-bundles by @dependabot[bot] in #949
  • chore(deps): bump requests from 2.32.3 to 2.32.4 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #950
  • chore(deps): bump lodash from 4.17.21 to 4.17.23 in /website by @dependabot[bot] in #952
  • chore(deps): bump apache-airflow from 3.1.0 to 3.1.6 in /data-stacks/airflow-on-eks/examples/dag-bundles by @dependabot[bot] in #955
  • feat: Add Pinot data stack by @nabuskey in #954
  • chore: Remove pinot blueprint by @nabuskey in #956
  • doc: Add documentation for NAU by @nabuskey in #958
  • fix: Fix trailing whitespace by @nabuskey in #967
  • fix: Fix home page links by @nabuskey in #962
  • fix: Fix box overflow by @nabuskey in #959
  • chore: Remove superset on eks blueprint by @nabuskey in #966
  • chore: Remove datahub on eks blueprint by @nabuskey in #965
  • chore: Remove spark ipv6 blueprint by @nabuskey in #964
  • fix: Update SHS memory allocation by @nabuskey in #976
  • fix: Add fernet key for airflow by @nabuskey in #977
  • chore: Upgrade karpenter by @nabuskey in #978
  • chore(deps): bump protobuf from 5.29.5 to 5.29.6 in /data-stacks/workshop/src/data-flow by @dependabot[bot] in #969
  • feat: Add celeborn best practices by @nabuskey in #980
  • feat: Add datafusion comet benchmark results by @nabuskey in #975
  • chore(deps): bump flask from 3.1.1 to 3.1.3 in /data-stacks/spark-on-eks/benchmarks/spark-operator-benchmarks by @dependabot[bot] in #982
  • chore(deps): bump flask from 3.1.1 to 3.1.3 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #984
  • chore(deps): bump werkzeug from 3.1.5 to 3.1.6 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #983
  • chore(deps): bump werkzeug from 3.1.5 to 3.1.6 in /data-stacks/spark-on-eks/benchmar...
Read more

v1.2.0

21 Jul 15:56
4468a88

Choose a tag to compare

What's Changed

  • feat: Prepare for v2.0 – New DoEKS Website, Deprecation of AI on EKS References, and Removal of Unused Blueprints by @vara-bonthu in #876

Full Changelog: v1.1.0...v1.2.0

v1.1.0

19 Jul 23:37
11f2e44

Choose a tag to compare

⚠️ Deprecation Notice: AI on EKS and Legacy Blueprints

As of this release, several legacy blueprints—including AI on EKS examples—remain available but will be deprecated in the upcoming v2.0 release of Data on EKS.

The upcoming v2.0 will focus exclusively on data-focused blueprints and best practices for running scalable data workloads on Amazon EKS.

If you’re looking for AI/ML workloads, model serving patterns, or blog content that references AI on EKS, please visit the dedicated AI on EKS site: 🔗 https://awslabs.github.io/ai-on-eks/

We encourage all users of AI blueprints to migrate to the AI on EKS repository for continued support and updates.


What's Changed

  • docs: Update and fix documentation for website doc by @ratnopamc in #658
  • docs: Fix a link in AWS Batch instructions by @delagoya in #659
  • feat: Update gradio client with sample prompt for Llama3.1-405b model inference by @ratnopamc in #662
  • feat: Spark operator default to encrypted gp3 PVs by @alanty in #674
  • feat: Use unique name for the spark job execution IRSA policy by @alanty in #672
  • refactor: Update Kafka Blueprint to use more recent versions of Kafka and Strimzi, and use Karpenter by @chrismld in #665
  • feat: BioNemo EKS cluster Using Cloudwatch Observability Plugin by @JohnDzialo in #641
  • chore(deps): bump http-proxy-middleware from 2.0.6 to 2.0.7 in /website by @dependabot[bot] in #681
  • feat: Spark benchmarks update to 3.5.3 by @vara-bonthu in #683
  • feat: Spark benchmarking updates with generic files by @vara-bonthu in #688
  • feat: Use official neuron helm chart for device plugin and scheduler by @askulkarni2 in #687
  • fix: Update EKS module to support AL2023 bootstrapping by @alanty in #689
  • fix: Add implicit dependecy for karpenter resources by @askulkarni2 in #691
  • chore: Update Spark v4 addons by @alanty in #692
  • feat: Spark benchmark observability by @alanty in #693
  • chore(deps): bump cookie and express in /website by @dependabot[bot] in #678
  • chore: Remove cluster-autoscaler examples for spark operator by @alanty in #695
  • fix: Add all VPC CIDRs to VPCe config by @alanty in #697
  • feat: Add spark benchmark test data generation changes by @ratnopamc in #694
  • docs: Update Spark Benchmark Docs by @ratnopamc in #698
  • docs: Add data generation doc updates for spark benchmark by @ratnopamc in #699
  • chore: Spark examples update by @alanty in #696
  • docs: Document how to run the TPCDS benchmarks by @alanty in #700
  • docs: Spark Benchmarks on Graviton R-series by @alanty in #701
  • style: Add cf insights by @candonov in #703
  • docs: Troubleshooting section for IPv6 policy by @alanty in #705
  • fix: Update spark ipv6 install.sh by @alanty in #704
  • feat: Trino blueprint upgrade by @vara-bonthu in #702
  • feat: Export Trino metrics to Prometheus and enable autoscalling with KEDA by @bbgu1 in #712
  • chore(deps): bump nanoid from 3.3.7 to 3.3.8 in /website by @dependabot[bot] in #714
  • chore(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /website by @dependabot[bot] in #710
  • chore: Corrected paths for install.sh/cleanup.sh commands in jark.md by @ak-org in #719
  • feat: Enhance Spark operator blueprint to add S3 Tables support by @ratnopamc in #721
  • docs: Add website documentation for using S3 Tables with Spark Operator by @ratnopamc in #722
  • feat: Add an example of using S3 tables with jupyter hub by @nabuskey in #723
  • fix: Flink operator blueprint deployment errors by @ratnopamc in #731
  • Update ADOPTERS.md by @vara-bonthu in #735
  • feat: Enable spark-operator prometheus metrics by @alanty in #737
  • feat: Add Karpenter and EMR Spark Dashboards to spark-operator by @alanty in #738
  • fix: Disable spark-operator podmonitor by @alanty in #742
  • docs: Clarifying where customers need the table bucket ARN vs Table ARN by @davlaur in #743
  • feat: Add Grafana dashboard for kubeflow Spark Operator by @ratnopamc in #744
  • feat: DeepSeek-R1 on EKS blueprint by @vara-bonthu in #745
  • chore(deps): bump aws-cdk-lib from 2.80.0 to 2.177.0 in /analytics/cdk/emr-eks by @dependabot[bot] in #733
  • feat: Fixes to security vulnerabilities by @vara-bonthu in #746
  • fix: Cleanup load balancers in spark operator cleanup by @alanty in #749
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc) by @e-mi-lio in #750
  • fix: Trino scale-down to zero with KEDA by @bbgu1 in #726
  • fix: Update Spark-operator kubecost by @alanty in #759
  • docs: NVIDIA triton server monitoring namespace fix by @ronaldosaheki in #765
  • feat: For issue #706 Ray serve with Llama.cpp for CPU inference on Graviton by @ddynwzh1992 in #739
  • feat: Add Spark operator benchmark kit by @alanty in #764
  • chore: Spark blueprint version bump by @alanty in #763
  • feat: Update EMR Spark RAPIDs blueprint by @ratnopamc in #762
  • feat: Spark operator Benchmarks update by @vara-bonthu in #769
  • feat: Add an example notebook for S3 Tables by @nabuskey in #771
  • fix: Add missing S3 tables permission by @ratnopamc in #776
  • feat: Spark observability improvements by @alanty in #778
  • chore(deps): bump @babel/runtime from 7.24.7 to 7.26.10 in /website by @dependabot[bot] in #787
  • chore(deps): bump @babel/helpers from 7.24.7 to 7.26.10 in /website by @dependabot[bot] in #786
  • chore(deps): bump @babel/runtime-corejs3 from 7.24.7 to 7.26.10 in /website by @dependabot[bot] in #785
  • chore(deps): bump prismjs from 1.29.0 to 1.30.0 in /website by @dependabot[bot] in #780
  • chore(deps): bump jinja2 from 3.1.5 to 3.1.6 in /analytics/terraform/spark-k8s-operator/examples/benchmark/spark-operator-benchmark-kit by @dependabot[bot] in #777
  • fix: Datahub version upgrade by @bbgu1 in #784
  • feat: Adding example for Fine-tuning Llama3 with LoRA using HuggingFace Optimum Neuron on AWS Trainium by @VijoyChoyi in #779
  • refactor: Updated adapter and hpa to use queue_compute_ratio by @freschri in #790
  • docs: Announce Data-on-EKS split and AI-on-EKS migration by @vara-bonthu in #794
  • chore(deps): bump aws-cdk-lib from 2.177.0 to 2.184.0 in /analytics/cdk/emr-eks by @dependabot[bot] in #795
  • fix: Update ingress-nginx by @askulkarni2 in #796
  • chore: Fix pre-commit errors by @askulkarni2 in #797
  • chore(deps): bump tar-fs in /website by @dependabot[bot] in #798
  • feat: Updates to AI on EKS repo by @vara-bonthu in #799
  • chore(deps): bump image-size from 1.1.1 to 1.2.1 in /website by @dependabot[bot] in #804
  • feat: Add Aerospike blueprint by @sud82 in #800
  • fix: Spark dashboard fixes by @alanty in #807
  • chore(deps): bump aws-cdk-lib from 2.184.0 to 2.187.0 in /analytics/cdk/emr-eks by @dependabot[bot] in #805
  • chore(deps): bump estree-util-v...
Read more

v1.0.4

19 Sep 19:52
4136b09

Choose a tag to compare

What's Changed

  • fix: Update cleanup script by @ratnopamc in #595
  • fix: Update karpenter version for trainium-inferentia blueprint by @ratnopamc in #599
  • docs: Add video link for Deploy accelerator-agnostic inference pipelines to Amazon EKS by @TalHibner in #600
  • feat: Replaced NIM architecture diagram with self-made NIM on EKS arch diagram by @hustshawn in #612
  • feat: RayServe with vLLM using AWS Neuron on Amazon EKS by @ratnopamc in #607
  • feat: Mountpoint S3 for loading additional Spark Jars by @bainskb in #606
  • feat: Support preloading container images into Bottlerocket data volumes with Karpenter by @lindarr915 in #587
  • chore: Delete ai-ml/kubeflow directory by @askulkarni2 in #619
  • feat: Updated mountpoint-s3 for spark readme by @bainskb in #618
  • feat: Trainium blueprint upgrade and Llama3.1 405b Distributed inference example by @vara-bonthu in #622
  • feat: Neuron scheduler update for trainium-inferentia blueprints by @ratnopamc in #624
  • feat: Website Updates by @vara-bonthu in #626
  • feat: Updates to the sidebar by @vara-bonthu in #627
  • feat: Added deprecating notes; added Jark stack doc;added warnings for ML p… by @vara-bonthu in #628
  • feat: NVIDIA NIM Updates by @vara-bonthu in #631
  • feat: Udate NVIDIA NIM blueprint with grafana dashboard and docs by @ratnopamc in #633
  • feat: Add OpenWebUI for vllm-rayserve-inf2 blueprint by @ratnopamc in #635
  • feat: Updated EMR on EKS Blueprint by @vara-bonthu in #638
  • chore: Update PULL_REQUEST_TEMPLATE.md by @askulkarni2 in #643
  • chore: Add access entry for workshop by @askulkarni2 in #644
  • fix: Use eks module for access_entries for trn-inf blueprint by @askulkarni2 in #646
  • chore(deps): bump send and express in /website by @dependabot in #653
  • chore(deps): bump serve-static and express in /website by @dependabot in #652
  • feat: Updates to the llama3.1 405 model scripts by @vara-bonthu in #655
  • feat: Add cloudwatch eks add on with enhanced monitoring for neuron by @ratnopamc in #651
  • chore: Update cmd-shell image to python3.11 by @askulkarni2 in #656
  • docs: Website documentation for vllm inferencing using rayserve on AWS Inferentia by @sindhupalakodety in #637
  • feat: Adding support for AWS Batch by @delagoya in #620
  • docs: Ray vLLM Inf2 website doc updates by @ratnopamc in #657
  • feat: Observability for RayServe and vLLM GPU by @shivam-dubey-1 in #642
  • feat: Add binpacking examples by @hitsub2 in #615
  • feat: Spark K8s Operator on EKS IPv6 cluster by @ovaleanu in #499

New Contributors

Full Changelog: v1.0.3...v1.0.4

v1.0.3

16 Jul 23:27
3cfd28c

Choose a tag to compare

What's Changed

Read more

v1.0.2

24 Jul 13:53
284a54e

Choose a tag to compare

What's Changed

Read more

v1.0.1 Final release for Version1

01 Mar 14:43
4da0fee

Choose a tag to compare

NOTE: This is the final release for Data on EKS Version1 blueprints(Final Tag v1.0.1). We will be updating the DoEKS blueprints with major changes and it will be published under Version2 onwards

What's Changed

New Contributors

Full Changelog: v1.0.0...v1.0.1

v0.1 DoEKS initial release

13 Nov 08:27
da17069

Choose a tag to compare

What's Changed

New Contributors

Full Changelog: https://github.com/awslabs/data-on-eks/commits/v0.1