NOTICE: FULL O2 Cluster Outage, January 3 - January 10th
O2 will be completely offline for a planned HMS IT data center relocation from Friday, Jan 3, 6:00 PM, through Friday, Jan 10
- on Jan 3 (5:30-6:00 PM): O2 login access will be turned off.
- on Jan 3 (6:00 PM): O2 systems will start being powered off.
This project will relocate existing services, consolidate servers, reduce power consumption, and decommission outdated hardware to improve efficiency, enhance resiliency, and lower costs.
Specifically:
- The O2 Cluster will be completely offline, including O2 Portal.
- All data on O2 will be inaccessible.
- Any jobs still pending when the outage begins will need to be resubmitted after O2 is back online.
- Websites on O2 will be completely offline, including all web content.
More details at: https://harvardmed.atlassian.net/l/cp/1BVpyGqm & https://it.hms.harvard.edu/news/upcoming-data-center-relocation
Two Factor Authentication FAQ
Â
Basic information
When will O2 require two factor authentication?
Since Monday, December 3rd, 2018, two factor authentication has been required to connect to O2 (o2.hms.harvard.edu
), any O2 login host (e.g. login01.o2.rc.hms.harvard.edu
), the transfer cluster (transfer.rc.hms.harvard.edu
), or orchestra.med.harvard.edu
(which simply redirects to the O2 cluster) from outside the HMS network. For a detailed list of where two factor authentication is required from, please reference the table here.
Why is two factor authentication now required for logging to O2?
Due to increased hacking attempts on O2, it is necessary to take increased security measures to protect user accounts and research data. We are following the industry standard of adding extra layer of authentication, as passwords can be compromised. Likewise, Harvard FAS' Odyssey cluster has required 2-factor authentication for a number of years. We are using the Duo application to manage the two factor authentication process.
What do I need to do to set up two factor?
You'll need to register your device (mobile phone is recommended) with Duo for HMS to be able to authenticate using security codes. You can receive these security codes through a phone call, a text message, or through the Duo mobile app. If you already use Duo for the HMS VPN, then no action is needed. However, if you only use Duo with Harvard Key (not HMS), then you will need to set up a separate HMS Duo profile. First, connect via ssh
to o2.hms.harvard.edu
like you normally would, and you will receive a unique web address to register for HMS Duo:
DESKTOP:~$ ssh abc123@o2.hms.harvard.edu
Password: [input your account password]
Please enroll at https://api-2f8a17a9.duosecurity.com/portal?code=443789224a30udsihdj8a&akey=DAIUI3JAB2FHDJKS7MJRQ0G
You can copy your unique web address from the terminal application paste into the address bar of your web browser (e.g. Google Chrome), and follow the prompts to register.
Please contact us at rchelp@hms.harvard.edu if you have any trouble setting up Duo.
I already have two-factor set up using Duo for Harvard Key. Can I use this for authentication on O2?
Unfortunately, no, we cannot use Harvard Key Duo profiles for two-factor authentication on O2. O2 cluster accounts use the HMS ID credentials, which are distinct from Harvard Key. To register your phone/device with Duo for HMS, you can ssh
to o2.hms.harvard.edu
, which will generate a web address that you can use to register for HMS Duo. See the previous answer for more detail.
How do I test logging in with two factor authentication to O2?
You can connect to o2.hms.harvard.edu
 to test your two factor authentication. If you have already enrolled in Duo for HMS, you will receive a "Duo two-factor login" message after entering your HMS ID password. You can select from the available methods (push, phone call, SMS) to authenticate by. Detailed instructions are located here. If you have not enrolled in Duo for HMS, you will receive a unique link to register through.
Please contact us at rchelp@hms.harvard.edu if you encounter any errors using two factor authentication.
How do I add a new device or modify existing 2FA settings?Â
You can connect to https://secure.med.harvard.edu and authenticate with your regular credentials. When prompted for the two factor authentication step you will also be presented (on the left side of the screen) with option to add a new device or modify existing settings. For detailed directions, visit here (https://harvardmed.service-now.com/kb_view_customer.do?sysparm_article=KB0010832).
I'm getting the error "Your account has been locked out due to excessive authentication failures." What do I do?
If you have more than 10 consecutive failed Duo login attempts, such as through a script or some other automated task, your Duo account will be locked. The account will automatically be re-enabled in 60 minutes (1 hour), assuming you stopped the script/process that is trying to login multiple times. Otherwise, the account will keep locking and unlocking continuously.
I've changed my phone number/gone overseas/lost my phone, and now I can't authenticate. Help!
If you only had the one device attached to your Duo configuration, you'll have to contact the service desk to get your access reset. You won't be able to get into your configuration without a valid device on record (you can't change two-factor settings without actually performing two-factor authentication), so IT will need to intervene. Simply send an email to itservicedesk@hms.harvard.edu with your situation (e.g. "I changed my phone number/went overseas and can't finish two-factor authentication, can you reset my configuration?") and they should be able to help you out.
It's worth noting that if that device was also how you accessed Harvard Key-locked services, for example, you may need to reset your access there as well (Duo associated with Harvard Key is not in the domain of HMS IT's services - that falls under the domain of Harvard University IT, or HUIT for short). If this is the case for you, you may also need to send an email to ithelp@harvard.edu with a similar message. If you are using Duo to access very specific services, it may be relevant to mention them in your emails. However, it is suggested that you refrain from mentioning O2 in emails to HUIT, or else you may be redirected to HMS IT, which is unlikely to solve that aspect of your problem. Similarly, refrain from mentioning Harvard Key-related services in tickets to HMS IT, or you may be redirected to HUIT.
To mitigate this issue in general, it is recommended that you add a second device with stable credentials to your device list, such as a tablet (e.g. a device with no phone number but with the Duo app installed) so that you can still get into your device management settings to modify things if your primary device changes.
Examples
ssh
On and after December 3rd, there is simply an additional step to connect via ssh to the O2 cluster at o2.hms.harvard.edu
, any O2 login hosts (e.g. login01.o2.rc.hms.harvard.edu
), or the transfer cluster at transfer.rc.hms.harvard.eduÂ
from outside of the HMS network. You can connect via ssh
as you normally would; after entering your HMS ID password you will now see a new message about "Duo two-factor login". This Duo message will prompt you to enter a passcode or to select which method to authenticate by. You can choose to authenticate via Duo Push (in the Duo mobile app), through a phone call, or by entering a passcode that is sent via SMS to your phone. After confirming your login by one of the previous methods, you will see the familiar "Welcome to O2 (Orchestra 2)!" message and will have finished logging into the cluster.
DESKTOP:~$ ssh abc123@o2.hms.harvard.edu
Password: [input your account password]
Duo two-factor login for abc123
Enter a passcode or select one of the following options:
1. Duo Push to XXX-XXX-1234
2. Phone call to XXX-XXX-1234
3. SMS passcodes to XXX-XXX-1234
Passcode or option (1-3): 1
Pushed a login request to your device...
Success. Logging you in...
Welcome to O2 (Orchestra 2)!
scp
/sftp
/rsync
Commands like scp
, sftp
, or rsync
will still work as expected when using the transfer cluster transfer.rc.hms.harvard.edu.
 Once you submit the scp
/sftp
/rsync
 command to transfer data, you will receive a notification via your default Duo method (e.g. push, phone, sms) to authenticate if you are connecting from outside the HMS network. If you don't have a default DUO Method setup, then all scp/sftp/rsync or similar non-interactive commands will just hang.Â
Please visit this page on how to setup default duo method. For more information on transferring data to/from O2, please reference this page.Â
DESKTOP:~$ sftp abc123@transfer.rc.hms.harvard.edu
Password: [input your account password]
# now Duo will contact you via your preferred method
# once you've successfully authenticated, you will be connected:
Connected to abc123@transfer.rc.hms.harvard.edu.
sftp>
Additionally, when using scp or sftp from the command line, you can also configure ssh
parameters as well. This is useful for if you need to set up keep-alives to hold the process open, for example (see below for graphical options):
For more keep-alive options, you can try some settings on this page (use them with -o
in the above command).
SFTP/SCP applications like Filezilla, WinSCP, and Transmit
Transferring data to or from O2 using the transfer cluster (transfer.rc.hms.harvard.edu) via an SFTP/SCP application like Filezilla will require two factor authentication from outside of the HMS network. For information on transferring data to/from O2, please reference this page.
When you connect to transfer.rc.hms.harvard.edu
with Filezilla or an equivalent program, your default Duo method will be used to authenticate. If Duo push is your preferred method, then you'll get a notification from the Duo app, or if a phone call is your preferred method, you'll get a call, etc. Once you have confirmed via Duo, you will be able to transfer files using your SFTP/SCP application.
In order to reduce the number of times you need to authenticate via Duo, you can enable the keep-alive option in your SFTP/SCP application. In FileZilla, this is located under Settings > Connection > FTP. Click the "Send FTP keep-alive commands" checkbox to enable sending keep-alive commands to retain your Filezilla connection for longer periods of time (and prevent needing to authenticate with Duo for your next transfer):
The equivalent option for enabling keep-alive commands in WinSCP is under the Advanced Site Settings menu, under Connection:
Transmit requires modification of several settings to reduce the number authentication requests via Duo. Under Preferences > Advanced, select "Try to keep idle connections alive", as well as Preferences > Advanced > Advanced Server Settings > select "Tickle server during long transfers". You can also increase the number of seconds specified in "Connect Timeout" under Advanced Server Settings.
Additionally, Transmit takes advantage of Mac's App Nap, which can extend your battery life by putting inactive applications into a suspended state where they aren't able to use system resources. If App Nap is enabled for Transmit and the application is put into the background, you will get a Duo two-factor authentication request once you move Transmit to the foreground and try to transfer files to or from O2. You can prevent Transmit from using App Nap by running in your local terminal:
Caveat:
It's worth noting that all of these tools (and probably more) will behave as expected only if you are doing typical drag-and-drop activities inside the open session. If you are a user that uses a file manager to make edits to files in order to circumvent having to use a terminal-based text editor, if you perform some action such as right-click → edit, then save your edits afterward to re-upload to O2, you WILL be prompted to authenticate every single time you do this. This is because these programs typically open brand new connections to deal with such operations, which the cluster interprets to be brand new log-ins. The workaround to this is obviously to only perform drag-and-drop operations, as these use the existing open connection.
Of particular note, MobaXterm will prompt you for re-authentication every time even though you are performing a drag-and-drop activity.
We are looking into potential configuration settings or alternate offerings that may change the above behavior, and will modify this page accordingly if anything is found.
If your workflow is impacted by this (e.g. a local edit/re-upload workflow), It is strongly recommended that you find some alternate means of file editing that you are comfortable with. nano
is a solid entry-level terminal-based text editor, but is much less feature-rich than other programs such as vim
or emacs
, which have non-trivial learning curves. If you absolutely must use the mouse in your editing, you can download certain text editors with Linux support directly to O2 (such as Sublime Text), and use them over an active X11 connection. You will experience some latency, especially as the file you are looking at increases in size, but you will be able to edit your file and have full mouse capabilities. If you have questions about this option, please contact us (see below) and we can assist you in setting this up in your local cluster environment).