AWS - CONvertible reserved instances - broken

Amazon is a powerhouse of a company. Massive household name. I am a 'Prime' member and oftentimes I buy things on Amazon even though they are more expensive just because it is easier.

Many non-webdev types are unaware that Amazon is also a massive player in the web services space with their Amazon Web Services (AWS) offerings. In fact as stated here they are:

big enough to be the fifth-largest business software company in the world

I utilise their Elastic Cloud Compute (EC2) offerings for my companies server needs.

Whilst many aspects of their offering are not suitable for me as an overall solution they are the best (on paper). For example, for a previous product I needed a lot of memory on my servers and easily expandable storage volumes. I had had previous issues with Digital Ocean and at the time Linode had not released its expandable volumes offering (and it kept breaking). Furthermore I made the (incorrect) assumption that with Amazon's size and scale it would be a relatively safe bet.

Initially I had considered using multiple providers and taking advantage of each of their respective strengths. Unfortunately one of the downsides with EC2 is that they charge for data in and out (to the internet) whereas other offerings tend to include that in the cost or at least give you an extremely significant amount of included bandwidth. This meant that having a database server hosted on EC2 and a web server at Linode would be cost prohibitive (Amazon offer different pricing structures for connections within its network).

Over the years I have used numerous providers and transitioning between them is always a chore. There is always a server dependency or tool that you forget to reinstall. Everything will seem to work but will secretly be blowing up in the background. With moving to Amazon I was intending to do so for the long haul.

Convertible Reserved Instances (CRI)

The TL;DR explanation of Amazon's basic offering (on demand) is that you pay per minute for the server specs you want. In practice you pay monthly, but the cost is equivalent to the minute price * the number of minutes in a month.

The other main consideration for me at least is their 'Reserved Instances' which essentially amount to prepaying for your server to reduce the costs.

The final consideration is the Convertible Reserved Instance which essentially means that you can convert between reserved instances. If for example (heavily simplified) you have one large memory focussed server and you now need three small general servers you can convert between the two. Except you can't (or couldn't) - thats the point of this post.

For a more detailled look at the pricing offerings take a look here.

Paying for support.

The worst thing about AWS is that you have to pay for support. If you are a small company and very rarely need support the minimum cost of $29 is somewhat prohibitive. Fortunately I have never need it in the 2 years I have used AWS and surely I am not going to need it to utilise some of their well documented primary functionality.. right? Wrong.

So. The process should be simple. Choose what you want to exchange for, click exchange, done. The process is/was: Choose what you want to exchange for, click exchange, get charged, nothing happens.

After this did not work multiple times over multiple days I went over to reddit to seek guidance. Fortunately for me an EC2 engineer saw my post and reached out to me. Unfortunately when I then explained the problem he ignored me. Great !

I contacted the 'basic' support and they decided that a fundamental problem with their core offering was nothing more than a 'General Issue' and did nothing about it. It took 2 weeks to get a generic 'we will pass this to our technical team' response all the while I was paying for a reserved instance that I had no use for and could not exchange for something that I would use.

Eventually I discerned that sending a message to support and then tweeting 'AWSSupport' telling them you had updated the ticket got them to reply a little quicker. Still, they didn't see this as being a significant issue even though they did acknowledge that the offering was fundamentally not working.

Convert to convert

Eventually they did get my r4-large instance to convert into 23 t2-nano instances. Why would you need 23 t2-nano instances you might ask? I did not.

Unfortunately another massive issue with EC2 (specifically reserved instances) is that as outlined here:

If the total value (upfront price + hourly price * number of remaining hours) of the new Convertible Reserved Instance is less than the total value of the exchanged Convertible Reserved Instance, AWS automatically gives you a quantity of instances in the Convertible Reserved Instance that ensures that the total value is the same or higher than that of the exchanged Convertible Reserved Instance.

I.E. Whilst they market CRIs with the fact that when they release newer (typically better) server types you can upgrade.. in practice it is less than simple. If I wanted to convert a r4-large to the newer (and cheaper) r5-large instance I would have to get two of them and pay the difference.

I queried this on reddit and got some helpful insight. Turns out that the EC2 community is significantly better at provided support and guiding users around the complexities of EC2 than the Amazon team themselves. Given the size of Amazon that is an absolute disgrace.

The alternative is to convert the r4-large into lots of small instances, and then convert some of them into what you want (paying a much smaller true-up cost).

So whilst I now had 23 t2-micro instances, 22 of which I could (in principle) convert to 1 r5-large instance.. this would still require conversions to actually work. The support team told me that the issues had been fixed. Of course they had not been.

Many tweets later. I achieved what I had wanted. I converted a convertible r4-large into an r5-large. It only took 3 weeks.

I eventually got Amazon to refund me for all the failed conversion charges. How kind /s. They gave me $20 as a gesture of good faith I think.. In the chaos of all the small conversion charges, refunds, and credit notes I can't actually work out if they gave me $20 or not. Even if they did, rather than being a gesture of good faith it is an arrogant slap in the face to the amount of time I spent battling to tell them of a product issue.

Respectfully the convertible aspect of convertible reserved instances is an absolute s**tshow. The fact that Amazon are slow in acknowledgment of issues like this and even slower in fixing them is worrying. Their attitude and approach absolutely stinks although I will acknowledge that when replies were forthcoming the support agents were polite albeit not neccesarily always helpful.

Yes, I acknowledge that AWS is targetted more towards massive businesses spending tens of thousands of dollars a day but in my opinion this is all the more reason why they should be thankful that an individual like me is (was) quietly pointing out their issues.

An explanation please.

I did request an explanation from the support team and eventually I (kinda) got one:

In regards to the reason why this issue occurred, the service team advised me that there were data inconsistency issues that were causing delays in subscription activation. The service team is aware of this issue and they actively working on fixing it.

So by the sounds of things it is not neccesarily even fixed now (which makes them telling me it was fixed 6 days ago all the more annoying).

They did apologize in the end though.

Again, I apologize for the experience that you had converting your RIs, I hope that you will have a more pleasant experience working on AWS in the future.

Still think I might stop ordering from Prime in a sort of pathetic protest :P

Be careful - they may be the fifth-largest business software company in the world but they still have their problems.