Updated: Sepetember 25, 2014, 2:00 PM PDT
With the news of a significant AWS reboot going out to many AWS customers, RightScale has been contacting many of our customers to ensure that they are prepared for the maintenance. We wanted to share some FAQs with the best information we have. We will update these FAQs as we get new or different information.
If you have additional FAQs you’d like us to try to answer, please tweet suggestions to @rightscale #awsreboot. For RightScale customers who need help with questions that are specific to their situations, please contact the normal RightScale support channels.
AWS Reboot FAQs
- When will the maintenance start and end?
- If I didn’t get a maintenance notice, am I OK?
- Which EC2 instance types are not affected?
- What regions and AZs are affected?
- Does this affect Linux and Windows?
- What other AWS services will be affected?
- What does the reboot entail?
- What is the specific timing and will all my impacted instances be rebooted at once?
- Can I relaunch instances now to get patched hosts?
- What should I do to prepare?
- Will I lose my data when the instance reboots?
- What’s the difference between relaunch and reboot?
- Is the reboot related to the Bash vulnerability that hit yesterday?
- Is RightScale affected by the planned maintenance and do you anticipate downtime?
When will the maintenance start and end?
The overall maintenance window is starting on September 26, 2014, at 2:00 UTC/GMT (September 25, 2014, at 7:00 PM PDT), and ending on September 30, 2014, at 23:59 UTC/GMT (September 30, 2014, at 4:59 PM PDT).
Your specific maintenance window will depend on the regions and AZs you use. Check the Events page in the AWS console for the schedule for your impacted instances. Note that you will need to check all of your AWS accounts.
If I didn’t get a maintenance notice, am I OK?
AWS sent out email notices starting on September 24. Per the EC2 support forum, all notifications have been sent and new notices will be generated just after the top of the hour for new instance launches. We recommend checking the Events page in the AWS console for notices on your impacted instances and the maintenance windows for the duration of the reboot period. Even if you do not have any maintenance notices, we would suggest closely monitoring all AWS deployments over the coming days.
Which EC2 instance types are not affected?
AWS has said that less than 10 percent of the EC2 fleet is impacted and that the following instance types will not be affected: T1, T2, M2, R3, and HS1. Note that even within the affected instance types, AWS has said that not all instances will need to be rebooted. Check your maintenance notices for the specific instance IDs that are impacted.
As an example, in our own RightScale accounts, we received notices for roughly 10-20 percent of all our instances. This obviously represents a higher percentage of instances within the impacted instance types. Among our customers (who tend to be larger consumers of AWS), we are finding the vast majority have instances that will be impacted. However, accounts that use only the smaller T1 and T2 instance types (which is what you get in the AWS Free Tier) may not be impacted at all.
Again, in the spirit of excess caution, we would suggest closely monitoring all instances over the coming days.
Does this impact Linux and Windows instances?
Yes. Note that there is a special patch for Windows Server 2012R2 that needs to be applied in advance of the maintenance window. See the AWS console for your specific instances.
What regions and AZs are affected?
All regions and AZs are affected. See above for info on which instances within those AZs will be rebooted.
What other AWS services will be affected?
RDS, ElastiCache, and RedShift will all be affected. There will be some downtime of these services during the reboot. If you use these services, check your maintenance notices for details.
What does the reboot entail?
This will be a controlled reboot. AWS reports that instances will stay on the same host with the same ephemeral disks and the same IP address. However, information in memory will be lost.
What is the specific timing and will all my impacted instances be rebooted at once?
AWS is staging the reboot process so that each region will have only one AZ patched each day. Additionally, AWS states that it will not patch instances in multiple regions at the same time within the same AWS account.
Can I relaunch instances now to get patched hosts?
If you relaunch an instance before the maintenance, you may — but are not guaranteed — to get an already-patched host. Normally, whenever our RightScale Ops team receives a maintenance notice regarding a specific set of instances, we relaunch them as soon as possible at our convenience so that by the time the maintenance windows arrives, our instances are already on hosts that have had the maintenance done. This time, due to the scale of the patching, there is not enough patched capacity available to guarantee this.
Despite this, we do suggest that you attempt to relaunch impacted instances and then wait a while and check the AWS console to see if the newly launched instances generated a new maintenance window. Per an AWS employee on the EC2 support forum, as of 9:57 AM Pacific Time on September 25, AWS is saying that the AWS console will be updated hourly with scheduled maintenance events, which should include your newly launched instances:
“I am pleased to report that since my last post, our team has built, tested, and deployed tooling that continuously rechecks all running instances for missing scheduled maintenance data and re-populates it as needed. Further, I can confirm that the scheduled maintenance lists for all customers in all public regions are now complete, and that these are currently being updated hourly just after the top of the hour. We are working to make this more close to real time; I'll re-post here if anything more substantive happens on this front.”
If you are unsuccessful in getting a patched host (for example, you get a new maintenance event after the top of the hour, you can retry later). You can also try to relaunch on instance types that AWS says will not be affected.
What should I do to prepare?
See information about preemptive relaunching above.
For instances where a short reboot is safe and acceptable, you don’t need to do anything: They will simply reboot during maintenance (and stay on the same host with the same ephemeral disks and the same IP address).
For databases, if you have set up the recommended master-slave configuration across AZs, you have the option to reboot the impacted database servers in their respective AZs ahead of the maintenance window in an attempt to get an instance that is already patched. If that is not successful, you can failover out of impacted AZs ahead of the maintenance window using the following approach:
- Check the AZ of your master and slave.
- Check your AWS console “Events” page for the maintenance time frame for your master and slave AZs.
- Clone a new slave DB in a new AZ.
- Adjust your master DB and slave DB as appropriate to avoid the maintenance windows and keep a master and slave DB running at all times.
If you do not have a master-slave configuration across AZs and it is critical that you have no downtime of your database, you may want to consider setting up a slave DB in another AZ ahead of the maintenance.
Will I lose my data when the instance reboots?
No. Data on the local instance store will persist after the reboot.
What’s the difference between relaunch and reboot?
An EC2 instance reboot is an instance life cycle operation equivalent to an OS reboot. The instance remains on the same host, ephemeral storage is still in place, and the IP address remains the same.
An instance relaunch involves replacing the running instance with a new one. Typically, users will launch a new instance, configure it correctly and then transfer traffic from the old instance to the new one via a DNS change or an EIP re-assignment. Once traffic has been transferred, the old instance is terminated.
Because the current instance is being replaced with a relaunch, ephemeral storage will be lost along with the memory stack. Also, the new instance may be on a different physical host from where the previous instance was running in the cloud.
RightScale customers using RightScale ServerTemplates™ can relaunch their servers with a single click.
Is the reboot related to the Bash vulnerability that hit yesterday?
AWS has said this is not due to the Bash vulnerability. It plans to provide more details on October 1, once the patching is complete.
However, you should address the Bash vulnerability also. We’ve provided guidance for RightScale customers using RightScale MultiCloud Images™ on how to do that.
Is RightScale affected by the planned maintenance and do you anticipate downtime?
We have received maintenance notices for our AWS account and we have been relaunching instances preemptively as described above. Many of our larger instances are M2s and therefore are not affected. As a result of our architecture and proactive work, we do not expect downtime for the RightScale Multi-Cloud Platform™ during the AWS reboot.