Skip to content

Commit 9f8894c

Browse files
srhopkinsSteven HopkinscloudpossebotBenbentwogithub-actions[bot]
authored
Adding support for opensearch domains (#144)
* adding initial support for opensearch domains * Auto Format * Fix var validation * Updating outputs * Auto Format * Convert log options to dynamic blocks * Removing restricition that prevents dev single instance instances * Adding anonymous_iam_actions var/support * Auto Format * Adding support for additional policies * Fixing additional_policy_documents var type * Auto Format * Ading policy sid * Auto Format * source_policy_documents * Adding overrides for policy statements * Auto Format * Removing additional policy statements * Auto Format * update readme * deduplicate domain * terraform fmt * update tflint * Update elasticsearch_domain.tf Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add new example, revert to join("" to avoid type changes during this update. * Update elasticsearch_domain.tf Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * cleanup unused provider * update readme and docs * update readme --------- Co-authored-by: Steven Hopkins <[email protected]> Co-authored-by: cloudpossebot <[email protected]> Co-authored-by: Benjamin Smith <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1 parent 1285ccd commit 9f8894c

File tree

10 files changed

+468
-348
lines changed

10 files changed

+468
-348
lines changed

README.md

Lines changed: 94 additions & 185 deletions
Large diffs are not rendered by default.

docs/terraform.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
|------|---------|
66
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
77
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.35.0 |
8-
| <a name="requirement_null"></a> [null](#requirement\_null) | >= 2.0 |
98

109
## Providers
1110

@@ -31,6 +30,8 @@
3130
| [aws_elasticsearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_policy) | resource |
3231
| [aws_iam_role.elasticsearch_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
3332
| [aws_iam_service_linked_role.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
33+
| [aws_opensearch_domain.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain) | resource |
34+
| [aws_opensearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain_policy) | resource |
3435
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
3536
| [aws_security_group_rule.egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
3637
| [aws_security_group_rule.ingress_cidr_blocks](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
@@ -50,11 +51,13 @@
5051
| <a name="input_advanced_security_options_master_user_name"></a> [advanced\_security\_options\_master\_user\_name](#input\_advanced\_security\_options\_master\_user\_name) | Master user username (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
5152
| <a name="input_advanced_security_options_master_user_password"></a> [advanced\_security\_options\_master\_user\_password](#input\_advanced\_security\_options\_master\_user\_password) | Master user password (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
5253
| <a name="input_allowed_cidr_blocks"></a> [allowed\_cidr\_blocks](#input\_allowed\_cidr\_blocks) | List of CIDR blocks to be allowed to connect to the cluster | `list(string)` | `[]` | no |
54+
| <a name="input_anonymous_iam_actions"></a> [anonymous\_iam\_actions](#input\_anonymous\_iam\_actions) | List of actions to allow for the anonymous (`*`) IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
5355
| <a name="input_attributes"></a> [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,<br>in the order they appear in the list. New attributes are appended to the<br>end of the list. The elements of the list are joined by the `delimiter`<br>and treated as a single ID element. | `list(string)` | `[]` | no |
5456
| <a name="input_auto_tune"></a> [auto\_tune](#input\_auto\_tune) | This object represents the auto\_tune configuration. It contains the following filed:<br>- enabled - Whether to enable autotune.<br>- rollback\_on\_disable - Whether to roll back to default Auto-Tune settings when disabling Auto-Tune.<br>- starting\_time - Date and time at which to start the Auto-Tune maintenance schedule in RFC3339 format. Time should be in the future.<br>- cron\_schedule - A cron expression specifying the recurrence pattern for an Auto-Tune maintenance schedule.<br>- duration - Autotune maintanance window duration time in hours. | <pre>object({<br> enabled = bool<br> rollback_on_disable = string<br> starting_time = string<br> cron_schedule = string<br> duration = number<br> })</pre> | <pre>{<br> "cron_schedule": null,<br> "duration": null,<br> "enabled": false,<br> "rollback_on_disable": "NO_ROLLBACK",<br> "starting_time": null<br>}</pre> | no |
5557
| <a name="input_automated_snapshot_start_hour"></a> [automated\_snapshot\_start\_hour](#input\_automated\_snapshot\_start\_hour) | Hour at which automated snapshots are taken, in UTC | `number` | `0` | no |
5658
| <a name="input_availability_zone_count"></a> [availability\_zone\_count](#input\_availability\_zone\_count) | Number of Availability Zones for the domain to use. | `number` | `2` | no |
5759
| <a name="input_aws_ec2_service_name"></a> [aws\_ec2\_service\_name](#input\_aws\_ec2\_service\_name) | AWS EC2 Service Name | `list(string)` | <pre>[<br> "ec2.amazonaws.com"<br>]</pre> | no |
60+
| <a name="input_aws_service_type"></a> [aws\_service\_type](#input\_aws\_service\_type) | The type of AWS service to deploy (`elasticsearch` or `opensearch`). | `string` | `"elasticsearch"` | no |
5861
| <a name="input_cognito_authentication_enabled"></a> [cognito\_authentication\_enabled](#input\_cognito\_authentication\_enabled) | Whether to enable Amazon Cognito authentication with Kibana | `bool` | `false` | no |
5962
| <a name="input_cognito_iam_role_arn"></a> [cognito\_iam\_role\_arn](#input\_cognito\_iam\_role\_arn) | ARN of the IAM role that has the AmazonESCognitoAccess policy attached | `string` | `""` | no |
6063
| <a name="input_cognito_identity_pool_id"></a> [cognito\_identity\_pool\_id](#input\_cognito\_identity\_pool\_id) | The ID of the Cognito Identity Pool to use | `string` | `""` | no |
@@ -86,7 +89,7 @@
8689
| <a name="input_encrypt_at_rest_enabled"></a> [encrypt\_at\_rest\_enabled](#input\_encrypt\_at\_rest\_enabled) | Whether to enable encryption at rest | `bool` | `true` | no |
8790
| <a name="input_encrypt_at_rest_kms_key_id"></a> [encrypt\_at\_rest\_kms\_key\_id](#input\_encrypt\_at\_rest\_kms\_key\_id) | The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key | `string` | `""` | no |
8891
| <a name="input_environment"></a> [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
89-
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
92+
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the user IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
9093
| <a name="input_iam_authorizing_role_arns"></a> [iam\_authorizing\_role\_arns](#input\_iam\_authorizing\_role\_arns) | List of IAM role ARNs to permit to assume the Elasticsearch user role | `list(string)` | `[]` | no |
9194
| <a name="input_iam_role_arns"></a> [iam\_role\_arns](#input\_iam\_role\_arns) | List of IAM role ARNs to permit access to the Elasticsearch domain | `list(string)` | `[]` | no |
9295
| <a name="input_iam_role_max_session_duration"></a> [iam\_role\_max\_session\_duration](#input\_iam\_role\_max\_session\_duration) | The maximum session duration (in seconds) for the user role. Can have a value from 1 hour to 12 hours | `number` | `3600` | no |

elasticsearch_domain.tf

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
#
2+
# Elasticsearch Domain
3+
#
4+
5+
resource "aws_elasticsearch_domain_policy" "default" {
6+
count = local.elasticsearch_enabled && (length(var.iam_authorizing_role_arns) > 0 || length(var.iam_role_arns) > 0) ? 1 : 0
7+
domain_name = module.this.id
8+
access_policies = join("", data.aws_iam_policy_document.default[*].json)
9+
}
10+
11+
resource "aws_elasticsearch_domain" "default" {
12+
count = local.elasticsearch_enabled ? 1 : 0
13+
domain_name = module.this.id
14+
elasticsearch_version = var.elasticsearch_version
15+
16+
advanced_options = var.advanced_options
17+
18+
advanced_security_options {
19+
enabled = var.advanced_security_options_enabled
20+
internal_user_database_enabled = var.advanced_security_options_internal_user_database_enabled
21+
master_user_options {
22+
master_user_arn = var.advanced_security_options_master_user_arn
23+
master_user_name = var.advanced_security_options_master_user_name
24+
master_user_password = var.advanced_security_options_master_user_password
25+
}
26+
}
27+
28+
ebs_options {
29+
ebs_enabled = var.ebs_volume_size > 0 ? true : false
30+
volume_size = var.ebs_volume_size
31+
volume_type = var.ebs_volume_type
32+
iops = var.ebs_iops
33+
throughput = var.ebs_throughput
34+
}
35+
36+
encrypt_at_rest {
37+
enabled = var.encrypt_at_rest_enabled
38+
kms_key_id = var.encrypt_at_rest_kms_key_id
39+
}
40+
41+
domain_endpoint_options {
42+
enforce_https = var.domain_endpoint_options_enforce_https
43+
tls_security_policy = var.domain_endpoint_options_tls_security_policy
44+
custom_endpoint_enabled = var.custom_endpoint_enabled
45+
custom_endpoint = var.custom_endpoint_enabled ? var.custom_endpoint : null
46+
custom_endpoint_certificate_arn = var.custom_endpoint_enabled ? var.custom_endpoint_certificate_arn : null
47+
}
48+
49+
cluster_config {
50+
instance_count = var.instance_count
51+
instance_type = var.instance_type
52+
dedicated_master_enabled = var.dedicated_master_enabled
53+
dedicated_master_count = var.dedicated_master_enabled ? var.dedicated_master_count : null
54+
dedicated_master_type = var.dedicated_master_enabled ? var.dedicated_master_type : null
55+
zone_awareness_enabled = var.zone_awareness_enabled
56+
warm_enabled = var.warm_enabled
57+
warm_count = var.warm_enabled ? var.warm_count : null
58+
warm_type = var.warm_enabled ? var.warm_type : null
59+
60+
dynamic "zone_awareness_config" {
61+
for_each = var.availability_zone_count > 1 && var.zone_awareness_enabled ? [true] : []
62+
content {
63+
availability_zone_count = var.availability_zone_count
64+
}
65+
}
66+
67+
dynamic "cold_storage_options" {
68+
for_each = var.cold_storage_enabled ? [true] : []
69+
content {
70+
enabled = var.cold_storage_enabled
71+
}
72+
}
73+
}
74+
75+
dynamic "auto_tune_options" {
76+
for_each = var.auto_tune.enabled ? [true] : []
77+
content {
78+
desired_state = "ENABLED"
79+
rollback_on_disable = var.auto_tune.rollback_on_disable
80+
maintenance_schedule {
81+
# Required until https://github.com/hashicorp/terraform-provider-aws/issues/22239 would be resolved
82+
start_at = var.auto_tune.starting_time == null ? timeadd(timestamp(), "1h") : var.auto_tune.starting_time
83+
duration {
84+
value = var.auto_tune.duration
85+
unit = "HOURS"
86+
}
87+
cron_expression_for_recurrence = var.auto_tune.cron_schedule
88+
}
89+
}
90+
}
91+
92+
node_to_node_encryption {
93+
enabled = var.node_to_node_encryption_enabled
94+
}
95+
96+
dynamic "vpc_options" {
97+
for_each = var.vpc_enabled ? [true] : []
98+
99+
content {
100+
security_group_ids = var.create_security_group ? [join("", aws_security_group.default[*].id)] : var.security_groups
101+
subnet_ids = var.subnet_ids
102+
}
103+
}
104+
105+
snapshot_options {
106+
automated_snapshot_start_hour = var.automated_snapshot_start_hour
107+
}
108+
109+
dynamic "cognito_options" {
110+
for_each = var.cognito_authentication_enabled ? [true] : []
111+
content {
112+
enabled = true
113+
user_pool_id = var.cognito_user_pool_id
114+
identity_pool_id = var.cognito_identity_pool_id
115+
role_arn = var.cognito_iam_role_arn
116+
}
117+
}
118+
119+
log_publishing_options {
120+
enabled = var.log_publishing_index_enabled
121+
log_type = "INDEX_SLOW_LOGS"
122+
cloudwatch_log_group_arn = var.log_publishing_index_cloudwatch_log_group_arn
123+
}
124+
125+
log_publishing_options {
126+
enabled = var.log_publishing_search_enabled
127+
log_type = "SEARCH_SLOW_LOGS"
128+
cloudwatch_log_group_arn = var.log_publishing_search_cloudwatch_log_group_arn
129+
}
130+
131+
log_publishing_options {
132+
enabled = var.log_publishing_audit_enabled
133+
log_type = "AUDIT_LOGS"
134+
cloudwatch_log_group_arn = var.log_publishing_audit_cloudwatch_log_group_arn
135+
}
136+
137+
log_publishing_options {
138+
enabled = var.log_publishing_application_enabled
139+
log_type = "ES_APPLICATION_LOGS"
140+
cloudwatch_log_group_arn = var.log_publishing_application_cloudwatch_log_group_arn
141+
}
142+
143+
tags = module.this.tags
144+
145+
depends_on = [aws_iam_service_linked_role.default]
146+
}

examples/opensearch_basic/main.tf

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
provider "aws" {
2+
region = "us-east-2"
3+
}
4+
5+
module "opensearch" {
6+
source = "../../"
7+
namespace = "eg"
8+
stage = "dev"
9+
name = "es"
10+
dns_zone_id = "Z14EN2YD427LRQ"
11+
security_groups = ["sg-XXXXXXXXX", "sg-YYYYYYYY"]
12+
vpc_id = "vpc-XXXXXXXXX"
13+
subnet_ids = ["subnet-XXXXXXXXX", "subnet-YYYYYYYY"]
14+
zone_awareness_enabled = "true"
15+
aws_service_type = "opensearch"
16+
elasticsearch_version = "OpenSearch_2.9"
17+
instance_type = "t3.small.search"
18+
instance_count = 4
19+
ebs_volume_size = 10
20+
iam_role_arns = ["arn:aws:iam::XXXXXXXXX:role/ops", "arn:aws:iam::XXXXXXXXX:role/dev"]
21+
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
22+
encrypt_at_rest_enabled = "true"
23+
kibana_subdomain_name = "kibana-es"
24+
25+
advanced_options = {
26+
"rest.action.multi.allow_explicit_index" = "true"
27+
}
28+
}

examples/opensearch_basic/versions.tf

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
terraform {
2+
required_version = ">= 1.3"
3+
4+
required_providers {
5+
aws = {
6+
source = "hashicorp/aws"
7+
version = ">= 4.0"
8+
}
9+
}
10+
}

0 commit comments

Comments
 (0)