OreBits Storage Platform
The OreBits Storage Platform is an on-premise storage appliance hosted by CCIT, and was created to accommodate larger data storage needs at Mines. OreBits is a “middle-of-the-road” solution, offering a balance between performance, protection and price.
If you would like more information or have questions, please see the Memorandum of Understanding below, or contact Research Support Services for help.
If you are interested in purchasing additional storage for your group, please open a helpdesk ticket with “OreBits Storage Platform” in the “other” category.
- On-premise storage solution (required by some government and many industry contracts).
- Accessible on or off campus.
- Ability to share folders with Mines Multipass users.
- Purchased in increments of a terabyte.
- Customizable options based on need (single-copy, replicates, backup)
- Utilize Globus.org to transfer large files via gridFTP (project in progress).
- Sync files across computers and share with external collaborators (future project).
- OreBits is an enterprise-level Oracle ZS3-2 storage appliance.
- Configured as a ZFS RAIDZ2 with 2 spare drives.
- Dual controller’s, power supplies, and network connections
- Active storage is directly accessible for read/write via NFS, CIFS, scp, sftp, and rsync from computational resources within the Mines environment, including login and compute nodes.
- To access active storage from outside of the Mines environment, it is necessary to use scp, sftp (encrypted), or rsync, CIFs (via VPN). Connections can be made through orebits_storage.mines.edu. Additionally, data transfers can also be made utilizing gridftp/Globus; Globus will almost always provide much higher throughput and resiliency than all other connection options and is the preferred protocol for any but the smallest transfers.
OreBits was designed as a “middle-of-the-road”, on-premise solution, offering a balance between performance, protection and price. Users only pay hardware and maintenance contract costs at a subscription rate of $9.93/TB/mo. OreBits will be charged as a yearly subscription beginning on July 1st. Accounts that are initiated after July 1st will be prorated until June 30th to ensure concurrent subscriptions.
If OreBits will be funded by a research grant, please contact ORA regarding proper appropriation of funds.
Memorandum of Understanding
Colorado School of MinesCCIT / Research SupportOreBits Storage Platform MOU
This is the memorandum of understanding (MOU) for OreBits. It includes both a Service Level Agreement (services provided to the customer) and an End User Agreement (terms agreed to by the customer).
2. DESCRIPTION OF SERVICE
The OreBits storage appliance is a cooperative funded model to support storage for the Mines research community.
2.1 Service Classes:
There is currently one service class of storage available:
- For data that is frequently written or read
- Directly accessible (read/write) from computational resources within the Mines environment
- Accessible from outside the Mines environment only via specific data transfer protocols (scp, sftp, CIFs, gridftp/GlobusOnline) through designated gateway nodes
- Not designed for highly I/O-intensive or massively parallel usage
- Incremental snapshots to mitigate impact of accidental deletion of files
2.2 Service Details:
We offer two specific storage services. Each customer project will be associated with one of these services. If additional services are required, please contact CCIT / Research Support for assistance. Storage space must be pre-purchased, with a minimum purchase of 1 TB for 1 year. In addition to these storage services, it is recommended that you have a local backup copy of critical data.
- Active – single copy on disk
- Appropriate for non-critical data
- Volumes are configured as a mirrored RAID
- Some protection against accidental file deletion due to local snapshots
- No protection against catastrophic failure of the primary storage system
- Active with replication – as above but with files replicated at least once a day to a disk system in a different location
- Appropriate for data that would be difficult to recreate
- Volumes are configured as a mirrored RAID
- Some protection against accidental file due to local snapshots
- Good protection against catastrophic failure of the primary storage system
- Good protection against loss of the primary data center
- Replication happens at least once per day
3. END-USER EXPECTATIONS
See Appendix A for the End User Agreement, which must be acknowledged by the researcher / PI before a project can be initiated.
3.2 User/Group Administration:
Every research group using OreBits should provide CCIT / Research Support with two official contacts: The Principal Investigator (PI) and an alternate Technical Point of Contact (POC) person. The PI must be a Mines faculty member. These two people are the only ones authorized to make changes to the set of users that comprise a group, to add or remove users on the access list, or to request other changes on behalf of the project.
Any change request that may impact a group will have to come through the PI or POC. Without such approval, CCIT / Research Support will not act on group-level requests. Change and service requests should be made through the Mines helpdesk ticketing system.
The PI and POC should verify their list of users on a yearly basis. Note that while Mines Multipass password may be terminated after a person leaves the university, his or her files will be retained in OreBits and their ownership will be transferred to the PI. This is done to ensure that shared files do not become unusable by a research group when one user account becomes inactive. The PI is encouraged to purge out-of-date data periodically. In the event that a PI leaves the university, the PI understands that the data may be purged to reclaim space.
OreBits relies on Mines Multipass credentials for access and authorization. All users must have a valid Mines Multipass account which ensures that all users are Mines faculty, students, staff, or affiliates. Note that a Mines faculty member may sponsor a non-Mines person as an affiliate, and then add them to the service. CCIT / Research Support cannot act as the sponsor.
Mines users pay actual media costs for disk space. Appendix B outlines the costs of the different services.
3.5 Ownership of Media:
The physical media that the data resides on belongs to CCIT and may not be removed from OreBits
4. SERVICE EXPECTATIONS
4.1 Support levels:
Two levels of service are available:
- Tier 1: Support is provided during business hours only, Monday through Friday, 8am to 5pm.
- Tier 2: Support is provided from Monday through Friday 8am to 5pm, including after-hours support on those weekdays with “best effort”. There is no guarantee of weekend or holiday support, though CCIT / Research Support will try to respond to incidents impacting the operation of the service with “best effort”.
CCIT / Research Support will operate OreBits as a hybrid between the “Tier 1” and “Tier 2” service levels. Customer-initiated support requests (e.g., group membership or permission changes) will be “Tier 1”. The backend storage systems and gateway nodes will be “Tier 2”.
While every reasonable and good faith effort will be made to ensure the integrity, reliability and availability of OreBits and of the files stored on it, access to data in OreBits may be affected by circumstances outside of the control of CCIT / Research Support.
4.2 Accessing storage:
Active storage is directly accessible for read/write via NFS, CIFS, scp, sftp, and rsync from computational resources within the Mines environment, including login and compute nodes.
To access active storage from outside of the Mines environment, it is necessary to use scp, sftp (encrypted), or rsync, CIFs (via VPN). Connections can be made through orebits.mines.edu. Additionally, data transfers can also be made utilizing gridftp/Globus; Globus will almost always provide much higher throughput and resiliency than all other connection options and is the preferred protocol for any but the smallest transfers.
4.3 Duration of Service:
In the event that CCIT / Research Support ceases to provide OreBits or any comparable resource, CCIT / Research Support will give at least 90-day advance notice. It will be the responsibility of the PI to transfer their data to other storage resources within that time window.
4.4 Amount of storage available:
The service will be grown as needed to address demand. CCIT / Research Support reserves the right to limit initial or additional allocations (on a temporary basis) to ensure the availability of the service to all who request it.
When it becomes more clear which services have the highest demand, CCIT / Research Support can purchase additional infrastructure to support those services.
A report detailing the project’s current storage usage will be emailed within 10 days after the end of each month to the PI and the POC.
At least annually, CCIT / Research Support will provide a list of users to all PIs and POCs showing the names and user IDs for every user who has access to the project’s data. The PI’s must affirm that those individuals should retain access and request the removal of access for individuals who no longer need it.
Planned maintenance of the CCIT / Research Support infrastructure, including OreBits hardware, will take place every second Thursday of the month at 5:30pm. CCIT / Research Support reserves the right to change the window, always with advance notice.
4.7 Change management:
CCIT / Research Support will make every effort to announce to all OreBits customers any significant changes to the system a minimum of 30 days in advance. Exceptions may be critical security updates and bug fixes that improve the stability of the system.
CCIT / Research Support will provide expected throughput performance targets for each service type; however, since OreBits is a shared infrastructure actual performance may vary depending on workload from other customers. When they become available, statistics on data performance and availability will be posted.
We are not able to process refunds or pro-rate any fees for any time lost due to repairs or maintenance events (planned or otherwise), nor for any datacenter-related down time.
5. HELP/SUPPORT REQUESTS
OreBits users may initiate support requests through the CCIT Help Desk ticketing system by emailing email@example.com. While CCIT / Research Support will make every effort to respond to and resolve support requests, those support requests which require domain-specific knowledge or expertise may not be able to be handled by CCIT / Research Support alone. In these cases, the support request may be forwarded to the POC for assistance.
End User Agreement
Colorado School of Mines OreBits Storage Use Agreement
As the PI, I understand that my use of and access to the digital storage facility known as the Colorado School of Mines OreBits shall be in accordance with all of the following stipulations:
____ The PI is responsible for ensuring data stored on the OreBits Platform complies with all institutional policies as well as all state and federal laws, including copyright, the Health Insurance Portability and Accountability Act (HIPAA), the Family Educational Rights and Privacy Act (FERPA), and the International Traffic in Arms Regulations (ITAR). Please contact CCIT Research Support for advisement of sensitive data
- Colorado School of Mines CCIT Policies:
- Colorado School of Mines Administrative Policies
- Health Insurance Portability and Accountability Act (HIPAA):
- Family Educational Rights and Privacy Act (FERPA):
- International Traffic in Arms Regulations (ITAR):
____ If any of the files that I store on the OreBits Platform are subject to one or more agreements with any Institutional Review Board (IRB), including but not limited to the IRB of Mines, then I will take full responsibility for ensuring full compliance with such agreement(s).
____ If I am collaborating with colleagues who are at institutions outside of the United States of America (that is, outside of both US states and US territories), then I will take full responsibility for ensuring that those colleagues do not access the OreBits themselves, but rather I and/or or other members of my team who are at US institutions will access the OreBits on behalf of the entire team.
____ I understand that, if and when I cease to be employed by and/or a student at an institution in the United States of America, then access to my files on OreBits will be available only to those of my collaborators who are employed by and/or students at US institutions.
____ If I am one of the Principal/Co-Principal investigators of a team, then I will take full responsibility for ensuring that any other members of the team are likewise in full compliance.
____ In the event that the OreBits Platform ceases providing storage or any comparable resource, then I will take full responsibility for transferring any and all relevant files to other storage resources, and in a timely manner.
____ I will take full responsibility for ensuring that I keep abreast of and comply with changes to any of the relevant laws, policies and circumstances described above.
Project name: ________________________
(should be specific to group or project; will be used in directory path; should not change over life of project; cannot include spaces. Good example: smith_mri. Bad example: labdata.)
|Service||Cost/TB/mo||TB (whole numbers)||Months (12)*||Total Cost|
|Active with replication||$19.86|
*Initial account setups will be prorated until June 30th to ensure concurrent subscriptions
Requested start date ______________
Project PI _______________________
- Email ____________________
- Dept _____________________
Project POC _____________________
- Email ____________________
- Dept _____________________
Usernames for access:
Instructions for Windows users:
- Open Windows Explorer, My Computer or My Documents.
- Locate the option to map a network drive. The method for doing this will vary with different versions of Windows. Typically, it will be either:
Options > Tools > Map Network Drive or This PC > Computer > Map Network Drive
- A “Map Network Drive” (or similar) panel should appear. The first item identified will usually be the “Drive” letter to be assigned on your system. This will be the virtual drive letter that you will later use to access your files on the remote domain. Usually the last available drive letter will show up. You can any other unassigned drive letter in the drop-down list.
- The second item requested is the “path” or “share” that tells your computer where to look on the network to find your files; this is the “full path to your files”. For Orebits user, this will be \\orebits.mines.edu\[share name] brackets removed
- If the “Reconnect at Login” box is checked, you should check or uncheck it as appropriate. For desktop machines constantly on the Mines network, this is a good choice. Laptops that travel off the Mines network are more problematic.
- Press the “OK” or “Finish” button.
- If the information you provided is correct and the drive is available, you will be prompted to enter your ADIT password (or username and password). Submit that information and press the “OK” button. You may need to enter your username as “ADIT\username” instead of just “username”– indicating the domain where your account is found (ADIT in this case).
- If the system validates your username/password, the drive letter you selected will now be mapped and you should be able to navigate to that virtual drive and access your files, just as if it were a local drive on your computer.
Instructions for macOS Users:
To access a remote drive using macOS, determine the path to the remote network drive. The methods noted above may be useful for discovering the path if it is not already known.
- Using the Mac Finder, click Go > Connect to Server.
- Type the server address in this format:
smb://orebits.mines.edu/[share name] brackets removed
- If you are successful, click the [+] icon at right to save the address permanently.
Note: If asked for login credentials, use those for the system to which you are trying to connect, not your local Mac username and password (unless, of course, they are the same).
Instructions for Linux Users:
There are many ways to mount network drives using a Linux system. Some Linux distributions have graphical clients for this purpose. Others require command-line solutions. In general, when mounting Windows shares, the “Samba” package will be used. For help with your particular Linux distribution, and a particular network file system, please submit a support request at Mines Help Center (https://helpdesk.mines.edu). Describe as completely as possible the variety of Linux you are using and the file system you are attempting to access. Or proceed using the hints found below.
For Linux users comfortable with the terminal or shell interface, the simplest method for accessing the files involves using the Samba package’s “smbclient” executable, along with the cifs-utils package. They may need to be installed using the package manager appropriate to your Linux distribution, and can be used as follows:
sudo smbclient -m SMB3 -U username -W adit //orebits.mines.edu/[share name] brackets removed
sudo mount.cifs –verbose -o username=joeuser@adit,vers=3.0 //orebits.mines.edu/[share name] brackets removed Replace the “username” with your campus username, “server” with the server in question (Hornet for Z: drive, Files for Y: drive, orebits.mines.edu for Orebits users), and “share” with the share that is given to you during account setup.
For Linux users more comfortable in the GUI environment, there are a variety of options, but the most commonly available is to use Nautilus (or “Files” in recent versions of Ubuntu). For most, this is by far the easiest way to connect to a Windows share in Linux.
Click on “Connect to Server” in the bottom of the left hand side bar (on Ubuntu 18.04, you will need to click on “Other Locations” first), and enter the server address in the format:
smb://orebits.mines.edu/[share name] brackets removed
Enter. You will be asked to supply your ADIT username, domain name (ADIT), and ADIT password (these are highly likely to be the same login credentials as your MultiPass credentials). You will also specify how long the system should remember your ADIT password (choose “Remember forever” for maximum convenience). Then click Connect and your ADIT directory on Hornet will now be available to you via the Nautilus file manager.
In summary, this process involves a lot of variation between different Linux distributions and configurations. Let us know if you have problems with the instructions above and we will try to help if your version of Linux is one with which we are familiar.