Improve the visibility of Amazon RDS cases and configuration with AWS Config and Amazon Fast Suite

Amazon Relational Database Service (Amazon RDS) is a managed service you should use to arrange, function, and scale a relational database within the cloud. As organizations deploy extra database cases throughout AWS accounts and AWS Areas, sustaining visibility of their increasing Amazon RDS fleet turns into more and more advanced. Understanding configurations, safety settings, backup methods, and compliance necessities throughout a distributed atmosphere requires entry to a number of accounts.

On this put up, we present you the right way to construct a centralized dashboard for monitoring Amazon RDS configurations throughout your group by utilizing AWS Config and Amazon Fast Suite. This answer delivers detailed insights throughout completely different areas, reminiscent of abstract metrics, backup configurations, safety posture, engine and help data, prolonged configurations, and useful resource tagging.

Answer overview

With a consolidated view, you’ll be able to view numerous elements of your Amazon RDS fleet, from high-level metrics reminiscent of complete occasion counts and engine distributions to detailed configuration parts reminiscent of encryption standing, backup retention intervals, and upkeep home windows. The next screenshot reveals instance metrics in a Fast Suite dashboard.

To ship complete visibility throughout your distributed database fleet, this weblog put up’s centralized Amazon RDS monitoring answer makes use of AWS Config as the first information assortment engine, working at the side of Amazon Easy Storage Service (Amazon S3), Amazon Athena, and AWS Glue. AWS Config captures detailed configuration snapshots of your Amazon RDS assets constantly throughout a number of accounts and Areas, storing this information in a centralized Amazon S3 bucket that you simply handle. Athena, powered by the AWS Glue Knowledge Catalog, helps environment friendly SQL-based querying of this configuration information by a custom-built dashboard interface. The whole answer is deployed by a streamlined AWS CloudFormation template and offers you quick, actionable insights into your international Amazon RDS deployments with out requiring advanced integration work.

AWS Config information adjustments to supported AWS assets as configuration objects in JSON format and delivers them to an Amazon S3 bucket inside every respective AWS account. For this answer to work, you have to designate an Amazon S3 bucket to gather the aggregated configuration. You want all of your AWS Config recorder setups which can be in several accounts and Areas to level to this designated Amazon S3 bucket. With this structure, AWS Config can constantly monitor Amazon RDS configurations throughout your whole AWS atmosphere whereas sustaining a single, consolidated repository of configuration information. The AWS Config snapshot assortment account turns into the central hub the place snapshots from linked accounts and Areas converge, offering the muse for AWS environment-wide visibility with out requiring direct entry to particular person account assets. The related steps are detailed on this put up’s “Stipulations” part.

Athena supplies a serverless question service that permits direct SQL-based evaluation of the AWS Config snapshots saved within the central Amazon S3 bucket. By utilizing the Knowledge Catalog, Athena interprets the construction of those configuration information with out requiring advanced extract, remodel, and cargo (ETL) processes. Word that the prepackaged launch stack implements these steps.

Partitioning in Athena improves querying effectivity by lowering the quantity of information scanned. It makes use of partition keys to focus on particular information subsets to spice up efficiency. When a brand new configuration snapshot is added to the designated Amazon S3 bucket, an AWS Lambda perform partitions the info by Area and date in order that queries entry solely the newest information. Word that the prepackaged launch stack additionally implements these steps.

The next diagram illustrates this weblog put up’s answer structure.

Stipulations

This put up creates a Fast Sight dashboard for Amazon RDS by utilizing AWS Config information. The put up doesn’t cowl the setup of the AWS Config atmosphere. You have to ensure that all the next AWS Config stipulations are met earlier than you proceed to implement the answer:

1. Arrange AWS Config throughout accounts and Areas in your AWS atmosphere the place you’ve gotten Amazon RDS cases. For extra data, see Getting Began with AWS Config within the AWS Config Developer Information. In the event you use AWS Organizations, you’ll be able to allow AWS Config centrally for all of your accounts and Areas by utilizing AWS Techniques Supervisor Fast Setup.
   
   Word: For this answer to work, you want a delegated Amazon S3 bucket to gather the aggregated configuration. All of your AWS Config recorder setups in several accounts and Areas should level to this designated Amazon S3 bucket.
2. Arrange Athena. For extra data, see Arrange, administrative, and programmatic entry.
3. Join an Amazon Fast Suite subscription in the identical AWS account by which you arrange Athena.

For detailed directions about establishing the atmosphere for AWS Config information, see Methods to question your AWS useful resource configuration states utilizing AWS Config and Amazon Athena and Visualizing AWS Config information utilizing Athena and Fast Suite.

Deploy the CloudFormation template

The CloudFormation template we’re offering for this answer creates the required AWS assets, together with an AWS Glue database and desk, Lambda capabilities for partition administration, and Fast Suite assets for visualization. For this step, use the account and area the place you’ve gotten your centralized S3 bucket. Full the next steps to deploy the CloudFormation template:

1. Navigate to the AWS CloudFormation console.
2. Select Create Stack
3. Obtain the quicksight_deployment_template.yaml template.
4. For Specify template, select Add a template file and add the quicksight_deployment_template.yaml
5. Select Subsequent.
6. Present the next required parameters:
   1. For ConfigAggregatorBucket, enter the designated Amazon S3 bucket identify the place AWS Config information from all accounts and Areas is aggregated.
   2. For AthenaResultBucket, enter the Amazon S3 bucket identify the place Athena question outcomes shall be saved.
   3. For QuickSightAnalysisAuthor, enter the Fast Suite username of the one who will handle the analyses and dashboards.
   4. For DataCollectionDB, enter the identify for the Athena database (the default is datacollectiondb).
   5. For ResourcePrefix, enter the prefix for created assets (the default is rdsinventory-).
7. (Non-obligatory) In case your Amazon S3 buckets are encrypted with AWS Key Administration Service (AWS KMS), present the next parameters:
   1. For ConfigAggregatorBucketKmsArn, enter the AWS KMS key Amazon Useful resource Title (ARN) for the AWS Config bucket.
   2. For AthenaBucketKmsArn, enter the AWS KMS key ARN for the Athena outcomes bucket.
8. Select Subsequent.
9. On the Evaluation web page, choose I acknowledge that AWS CloudFormation may create IAM assets with {custom} names and select Create stack.

The template creates:

• An AWS Glue database and desk for storing Amazon RDS configuration information.
• A Lambda perform to handle AWS Glue partitions.
• An Amazon EventBridge rule to set off the Lambda perform.
• A Fast Suite information supply and dataset configurations.
• The required AWS Id and Entry Administration (IAM) roles and permissions.
• A Fast Suite dashboard.

The stack creation sometimes takes 5–10 minutes.

Confirm the answer

It’s best to confirm that the answer is in place by creating an AWS Config snapshot by the AWS CloudShell (AWS CloudShell). This may set off the info partition workflow and make the snapshot information out there as a dataset in Fast Suite for additional evaluation. You possibly can carry out this check in an account or Area that has AWS Config snapshot supply enabled to the centralized bucket.

Comply with these steps to create an AWS Config snapshot:

1. Use AWS CloudShell to create the AWS Config snapshot on one of many linked accounts that hosts Amazon RDS cases:

   aws configservice deliver-config-snapshot --delivery-channel-name 

2. In case you are not sure of the supply channel identify on your given account and Area, run the next command to checklist your supply channels:

   aws configservice describe-delivery-channels

   This may return the next output itemizing your supply channel identify:

   {
       "DeliveryChannels": [
           {
               "name": "",
               "s3BucketName": "",
               "snsTopicARN": "arn:aws:sns:::",
               "configSnapshotDeliveryProperties": {
                   "deliveryFrequency": "Three_Hours"
               }
           }
       ]
   }

3. From the output, s3BucketName is similar as your centralized Amazon S3 bucket that shops AWS Config snapshots. Word the worth for identify, which is your supply channel—s3BucketName isn’t your supply channel. The entire question to create an AWS Config snapshot will appear like the next code:

   aws configservice deliver-config-snapshot --delivery-channel-name 

4. Sign up to the info assortment account the place you’ve gotten your centralized S3 bucket and navigate to the Fast Suite console.
5. Within the Fast Suite console, select Datasets within the navigation pane.
6. Select rdsinventory-–.
7. On the Refresh tab, select REFRESH NOW.
   
   
8. Select Full refresh, after which select Proceed.
9. For Affirm refresh, select Refresh.
10. Return to the Fast Suite console dwelling web page and navigate to the Analyses web page.
11. Open the evaluation named rdsinventory-–. This may present you the dashboard giving visible details about your RDS cases throughout completely different accounts and areas.

Cleanup

To keep away from incurring future costs, delete the CloudFormation stack you used to deploy the answer assets. For directions, see Delete a stack from the CloudFormation console.

Conclusion

On this put up, we confirmed the right way to construct a centralized dashboard for monitoring your Amazon RDS fleet throughout a number of AWS accounts and Areas by utilizing AWS Config, Amazon S3, Athena, and Fast Suite. By utilizing AWS Config to seize detailed configuration snapshots from distributed accounts, storing these snapshots in a central Amazon S3 bucket, and utilizing Athena with AWS Glue to question this information effectively, you’ll be able to create complete visualizations in Fast Suite that present quick insights into your whole database panorama. This answer alleviates the necessity to entry a number of accounts individually, providing you with a single dashboard to grasp configurations, safety settings, backup methods, and compliance necessities throughout your AWS atmosphere’s full Amazon RDS deployment.

This answer avoids guide monitoring, reduces operational overhead, and helps you proactively handle your Amazon RDS cases. Now you can observe essential configurations, implement compliance with safety requirements, and make data-driven selections about your database infrastructure. The automated nature of this put up’s answer offers you entry to present, correct details about your Amazon RDS deployments to be able to keep operational excellence whereas scaling your database infrastructure.

Concerning the authors