All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Update aws provider version to 5.17.0
- Added
metastore-uriproperties tobeekeeper-scheduler-apiary-config.json.
- Added new variable
beekeeper_db_external_hostnameto support external DB host.
- Kubernetes variables to adjust number replicas on deployments.
- Added datadog annotations to send metrics from
scheduler-apiary,metadata-cleanup,path-cleanupto DD.
- Add
copy_tags_to_snapshotto aws_rds_cluster.
- Change kiam auth method to irsa for metadata cleanup, path cleanup and scheduler apiary
- Change provider version for kubernetes to 2.13.0
- Updating
beekeeper_slack_notifierLambda function inlambda.tfthat is currently using the deprecated Python 3.7 runtime to the Python 3.9 runtime
- Updated hosted runner from the deprecated
ubuntu-18.04toubuntu-22.04to fix failing GitHub Actions
- Added
HADOOP_USER_NAMEenvironment variable and set it tobeekeeper
- Upgdate k8s API to work with 2.x version:
kubernetes_deploymenttokubernetes_deployment_v1kubernetes_ingresstokubernetes_ingress_v1
- Update terrafrom
mapfunction totomap(mapfunction has been deprecated in0.12version).
- Increase k8s provider version to
~>2.7.0(was~>1.0.0). Fixes issue with K8S upgrade and error: 'Failed to create Ingress'.
- Added variables
db_apply_immediately(defaultfalse) anddb_performance_insights_enabled(defaulttrue). - Minimum required version for
awsprovider set to~> 4.0.
- Increased the required version of terraform to
>= 0.12.31(was> 0.12.0).
- Changed the
rds_storage_typefromgp2togp3. - Increased the
rds_allocated_storagefrom 10 to 20 GB.
- Changed the service namespace from
defaulttobeekeeperinbeekeeper-metadata-cleanup,beekeeper-path-cleanupandbeekeeper-scheduler-apiary. - Changed some variables from type
stringtonumber. - Fixed the
beekeeperingress so that thek8s_ingress_enabledvariable works.
- Terraform files for
beekeeper-apimodule.
- Pin K8S provider version to
1.xfor compatibility with other Apiary components deployed in the same Terraform state file. - Don't use instance alias as part of default DB name since all the Flyway scripts expect "beekeeper" as the db name.
- Removed variable
DB_PASSWORD_STRATEGY. - Securely injecting
db_password_keyin k8s and ecs instead of pulling these credentials from AWS Secret Manager in Beekeeper.
- Added variable
docker_registry_secret_nameto pull docker images from a private registry.
- Separated
dry_run_enabledvariable intopath_cleanup_dry_run_enabledandmetadata_cleanup_dry_run_enabledso one module can be used in dry run mode without affecting the other.
- Kubernetes deployment options for new Metadata Cleanup module.
- Renamed
CleanuptoPath Cleanup, andPath SchedulertoScheduler Apiaryto match name changes in Beekeeper.
- Fix multi instance deployment resource names.
- Default blank value for
k8s_kiam_role_arn.
- Refactoring graphite configuration so that it can be switched off.
- Add annotations to Kubernetes deployments to support Prometheus scraper.
- Add support for Kubernetes by specifying
var.instance_type = "k8s". - Add Kubernetes deployment options for Cleanup and Path Scheduler.
- Add KIAM role creation for k8s pods.
- Refactored to remove
mainandlambdamodules. - Updated Terraform syntax to be compliant with Terraform 0.12 and not throw warnings.
- Run
terraform fmton all code to enforce style.
- Updating S3 IAM policy name to be region specific.
- First version of Beekeeper terraform module