Cost estimation of usage-based resources (e.g. AWS S3/Lambda) #985
Replies: 10 comments 20 replies
-
Hi, According to sources, I'd prefer Cloudwatch also, because billing data is not always visible to devs and in larger organizations, it's centralized through billing accounts. Although billing data would be the most accurate source including discounts. |
Beta Was this translation helpful? Give feedback.
-
I definitely like the idea of being more accurate in the cost estimate. However, some customers may not want to allow direct access to their AWS account. If this feature is developed/added (which I hope it does), please also provide an "offline" method (ie. a CSV export) as an alternative. |
Beta Was this translation helpful? Give feedback.
-
I've just recently been doing scaling test runs for a whole day and using cost explorer to estimate pricing (with the intent of rolling the values back into the usage YAML) so this feature is quite timely for me. Syncing usage directly from AWS would be great, however, please can we allow it to estimate those usages using a date range? That means we can do things like only run a really large scale test for just one whole day, and scale back down keeping costs far, far lower than having to do a run for a whole 30 days! |
Beta Was this translation helpful? Give feedback.
-
will be awesome if we could also do this in GCP 🙏 |
Beta Was this translation helpful? Give feedback.
-
One of the community members has been hacking the atlantis diff script so the |
Beta Was this translation helpful? Give feedback.
-
I think there could be 3 possibilities here:
|
Beta Was this translation helpful? Give feedback.
-
This is a great feature, thanks for looking into it. However, it would be useful to know what permissions are required for this to work. I gave our user read only cloudwatch permissions but it seems like it needs ec2, s3 & dynamoDB permissions also. |
Beta Was this translation helpful? Give feedback.
-
This is a long comment, but I want to put it out there to see what people think about the alternative approach of using the AWS Cost and Usage Reports and equivalents in other clouds. Use casesWe're seeing two use cases for syncing usage data: 1. Cost visibility for all usage-based resources in CI Instead of seeing “Cost depends on usage” we can show the costs based on the last 30 days of usage. So when you make a change it doesn’t just show you the price for resources like S3 and Lambda, but the actual effect the change will have on your bill. 2. What-if analysis in the CLI Using your actual current usage as a baseline, you could see how your bill changes if your usage increases by a factor of 10, or if you switch to different infrastructure options, e.g. different S3 storage classes, you can see how this will affect your bill based on your past usage. Problems with current CloudWatch approachThe current experimental This approach has various challenges:
Using the AWS Cost and Usage Reports insteadOne idea we have is to connect to and ingest AWS Cost and Usage Reports (or equivalents for GCP and Azure) and sync the usage data from them. This will give coverage for all resources and allow us to map the usage based on the resource IDs in these files. Since the data is in a consistent format, we will be able to support all resources quickly and the usage data will match the bill as much as is possible. Thoughts?It would be great to get some input on this from the community. Particularly we want to know:
|
Beta Was this translation helpful? Give feedback.
-
This is a follow up on the previous comment since we're starting to look more closely at generating usage estimate from AWS Cost and Usage Reports. We'd like to get a sense of how people have their CURs setup, so it would be great if people could reply with:
Thanks! |
Beta Was this translation helpful? Give feedback.
-
👋 Hi everyone, we've released Actual Costs into private beta where Infracost can connect to your AWS CURs and other APIs to fetch Savings Plans and RIs. Please email me ali.hosseini@infracost.io if you'd like to work with us to enable this and discuss the options to expose usage-based costs in PR comments. |
Beta Was this translation helpful? Give feedback.
-
Status quo
We call resources such as AWS S3 or Lambda "usage-based resources" as their costs depend on usage, and for them the CLI outputs prices, e.g. "Cost depends on usage: $0.20 per 1M requests". The CLI enables you to describe usage estimates in a file called
infracost-usage.yml
so you can enter things like storage GB estimates and get a cost estimate (instead of just seeing the price).As of Infracost CLI
v0.9.8
, we're experimenting with fetching usage file values from CloudWatch.Over to you
What do you think about how Infracost is handling usage-based resources? Does it help your use-case? Should we consider other data sources such as the billing CSVs (e.g. AWS Cost & Usage Reports) or do those require access that engineers don't usually have?
Beta Was this translation helpful? Give feedback.
All reactions