Changes for Deploying Dashboards for a subset of accounts#434
Changes for Deploying Dashboards for a subset of accounts#434Lavanya0513 wants to merge 5 commits into
Conversation
|
|
||
| # --- SOURCE PATH RESOURCES --- | ||
| # S3 Bucket for Athena query results | ||
| AthenaResultsBucket: |
There was a problem hiding this comment.
Customers already have Athena query results bucket delivered with main DataExport template in data collection account
There was a problem hiding this comment.
Removed the Athena and Glue components. will go with cid cmd
| "billing_period" | ||
|
|
||
| FROM {source_db}.{source_table} | ||
| WHERE line_item_usage_account_id IN ('{account_filter}') |
There was a problem hiding this comment.
Add a validation that account ids are matching account id format ^\d{12}$
| - athena:GetQueryExecution | ||
| - athena:GetQueryResults | ||
| - athena:StopQueryExecution | ||
| Resource: "*" |
There was a problem hiding this comment.
the Lambda role should be scoped to specific workgroups, databases, and tables rather than *
There was a problem hiding this comment.
Restricted to primary workgroup
| - Sid: CrossAccountAccess | ||
| Effect: Allow | ||
| Principal: | ||
| AWS: !Sub 'arn:aws:iam::${DataCollectionAccountId}:root' |
There was a problem hiding this comment.
Granting :root means ANY principal in that account (any role, user, or service) can read, write, and delete objects. This should be scoped to a specific role ARN. The s3:DeleteObject permission is particularly concerning for a data collection bucket.
|
|
||
| Resources: | ||
| # --- TARGET PATH RESOURCES --- | ||
| TargetDataBucket: |
There was a problem hiding this comment.
Should enforce SSE-S3 or SSE-KMS at minimum
| print(f"Started query execution: {query_execution_id}") | ||
|
|
||
| # Wait for query completion | ||
| max_wait_time = 600 # 10 minutes |
There was a problem hiding this comment.
What happens to the query if it exceeds 10 min?
There was a problem hiding this comment.
Added a env variable. if Lambda times out, user can increase this value. Will add in instructions.
|
|
||
| # Delete all existing files under data directory | ||
| prefix = f'cur2/{payer_account}/cid-cur2/cid-cur2/' | ||
| delete_all_files(target_bucket, prefix) |
There was a problem hiding this comment.
What if Athena query fails afterwards? Will the bucket end up being empty?
2 participants
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.