OreBits Storage Platform

Graphical Version

OreBits Storage Platform


Overview

The OreBits Storage Platform is an on-premise storage appliance hosted by CCIT, and was created to accommodate larger data storage needs at Mines.  OreBits is a "middle-of-the-road" solution, offering a balance between performance, protection and price.

If you would like more information or have questions, please see the Memorandum of Understanding below, or contact Research Support Services for help.

If you are interested in purchasing additional storage for your group, please open a helpdesk ticket with "OreBits Storage Platform" in the "other" category.

 


Features

  • On-premise storage solution (required by some government and many industry contracts).
  • Accessible on or off campus.
  • Ability to share folders with Mines Multipass users.
  • Purchased in increments of a terabyte.
  • Customizable options based on need (single-copy, replicates, backup)
  • Utilize Globus.org to transfer large files via gridFTP (project in progress).
  • Sync files across computers and share with external collaborators (future project).

Hardware:

  • OreBits is an enterprise-level Oracle ZS3-2 storage appliance.
  • Configured as a ZFS RAIDZ2 with 2 spare drives.
  • Dual controller's, power supplies, and network connections

Connectivity:

  • Active storage is directly accessible for read/write via NFS, CIFS, scp, sftp, and rsync from computational resources within the Mines environment, including login and compute nodes.
  • To access active storage from outside of the Mines environment, it is necessary to use scp, sftp (encrypted), or rsync, CIFs (via VPN).  Connections can be made through orebits_storage.mines.edu.  Additionally, data transfers can also be made utilizing gridftp/Globus; Globus will almost always provide much higher throughput and resiliency than all other connection options and is the preferred protocol for any but the smallest transfers.  

Billing

OreBits was designed as a "middle-of-the-road", on-premise solution, offering a balance between performance, protection and price.  Users only pay hardware and maintenance contract costs at a subscription rate of $9.93/TB/mo.  OreBits will be charged as a yearly subscription beginning on July 1st. Accounts that are initiated after July 1st will be prorated until June 30th to ensure concurrent subscriptions.

If OreBits will be funded by a research grant, please contact ORA regarding proper appropriation of funds.

Memorandum of Understanding

 

Colorado School of Mines
CCIT / Research Support
OreBits Storage Platform MOU

 

 

1.  INTRODUCTION

This is the memorandum of understanding (MOU) for OreBits.  It includes both a Service Level Agreement (services provided to the customer) and an End User Agreement (terms agreed to by the customer). 

2.  DESCRIPTION OF SERVICE

The OreBits storage appliance is a cooperative funded model to support storage for the Mines research community.

2.1 Service Classes:

There is currently one service class of storage available: 

  • Active
    • For data that is frequently written or read
    • Directly accessible (read/write) from computational resources within the Mines environment
    • Accessible from outside the Mines environment only via specific data transfer protocols (scp, sftp, CIFs, gridftp/GlobusOnline) through designated gateway nodes
    • Not designed for highly I/O-intensive or massively parallel usage
    • Incremental snapshots to mitigate impact of accidental deletion of files

2.2 Service Details:

We offer two specific storage services.  Each customer project will be associated with one of these services.  If additional services are required, please contact CCIT / Research Support for assistance.  Storage space must be pre-purchased, with a minimum purchase of 1 TB for 1 year.  In addition to these storage services, it is recommended that you have a local backup copy of critical data.

  • Active - single copy on disk
    • Appropriate for non-critical data
    • Volumes are configured as a mirrored RAID
    • Some protection against accidental file deletion due to local snapshots
    • No protection against catastrophic failure of the primary storage system
  • Active with replication - as above but with files replicated at least once a day to a disk system in a different location
    • Appropriate for data that would be difficult to recreate
    • Volumes are configured as a mirrored RAID
    • Some protection against accidental file due to local snapshots
    • Good protection against catastrophic failure of the primary storage system
    • Good protection against loss of the primary data center
    • Replication happens at least once per day

3.  END-USER EXPECTATIONS

3.1 Usage:

See Appendix A for the End User Agreement, which must be acknowledged by the researcher / PI before a project can be initiated. 

3.2 User/Group Administration:

Every research group using OreBits should provide CCIT / Research Support with two official contacts: The Principal Investigator (PI) and an alternate Technical Point of Contact (POC) person.  The PI must be a Mines faculty member.  These two people are the only ones authorized to make changes to the set of users that comprise a group, to add or remove users on the access list, or to request other changes on behalf of the project. 

Any change request that may impact a group will have to come through the PI or POC.  Without such approval, CCIT / Research Support will not act on group-level requests.  Change and service requests should be made through the Mines helpdesk ticketing system. 

The PI and POC should verify their list of users on a yearly basis.  Note that while Mines Multipass password may be terminated after a person leaves the university, his or her files will be retained in OreBits and their ownership will be transferred to the PI.  This is done to ensure that shared files do not become unusable by a research group when one user account becomes inactive.  The PI is encouraged to purge out-of-date data periodically.  In the event that a PI leaves the university, the PI understands that the data may be purged to reclaim space.

OreBits relies on Mines Multipass credentials for access and authorization.  All users must have a valid Mines Multipass account which ensures that all users are Mines faculty, students, staff, or affiliates.  Note that a Mines faculty member may sponsor a non-Mines person as an affiliate, and then add them to the service.  CCIT / Research Support cannot act as the sponsor. 

3.4 Costs/Fees:

Mines users pay actual media costs for disk space.  Appendix B outlines the costs of the different services. 

3.5 Ownership of Media:

The physical media that the data resides on belongs to CCIT and may not be removed from OreBits

4.  SERVICE EXPECTATIONS

4.1 Support levels:

Two levels of service are available: 

  • Tier 1: Support is provided during business hours only, Monday through Friday, 8am to 5pm.
  • Tier 2: Support is provided from Monday through Friday 8am to 5pm, including after-hours support on those weekdays with “best effort”. There is no guarantee of weekend or holiday support, though CCIT / Research Support will try to respond to incidents impacting the operation of the service with “best effort”. 

CCIT / Research Support will operate OreBits as a hybrid between the “Tier 1” and “Tier 2” service levels.  Customer-initiated support requests (e.g., group membership or permission changes) will be “Tier 1”.  The backend storage systems and gateway nodes will be “Tier 2”.

While every reasonable and good faith effort will be made to ensure the integrity, reliability and availability of OreBits and of the files stored on it, access to data in OreBits may be affected by circumstances outside of the control of  CCIT / Research Support. 

4.2 Accessing storage:

Active storage is directly accessible for read/write via NFS, CIFS, scp, sftp, and rsync from computational resources within the Mines environment, including login and compute nodes.

To access active storage from outside of the Mines environment, it is necessary to use scp, sftp (encrypted), or rsync, CIFs (via VPN).  Connections can be made through orebits.mines.edu.  Additionally, data transfers can also be made utilizing gridftp/Globus; Globus will almost always provide much higher throughput and resiliency than all other connection options and is the preferred protocol for any but the smallest transfers. 

4.3 Duration of Service:

In the event that CCIT / Research Support ceases to provide OreBits or any comparable resource, CCIT / Research Support will give at least 90-day advance notice.  It will be the responsibility of the PI to transfer their data to other storage resources within that time window. 

4.4 Amount of storage available:

The service will be grown as needed to address demand.  CCIT / Research Support reserves the right to limit initial or additional allocations (on a temporary basis) to ensure the availability of the service to all who request it.

When it becomes more clear which services have the highest demand, CCIT / Research Support can purchase additional infrastructure to support those services. 

4.5 Reporting:

A report detailing the project’s current storage usage will be emailed within 10 days after the end of each month to the PI and the POC. 

At least annually, CCIT / Research Support will provide a list of users to all PIs and POCs showing the names and user IDs for every user who has access to the project’s data.  The PI’s must affirm that those individuals should retain access and request the removal of access for individuals who no longer need it.

4.6 Maintenance:

Planned maintenance of the CCIT / Research Support infrastructure, including OreBits hardware, will take place every second Thursday of the month at 5:30pm.  CCIT / Research Support reserves the right to change the window, always with advance notice. 

4.7 Change management:

CCIT / Research Support will make every effort to announce to all OreBits customers any significant changes to the system a minimum of 30 days in advance.  Exceptions may be critical security updates and bug fixes that improve the stability of the system.

4.8 Performance:

CCIT / Research Support will provide expected throughput performance targets for each service type; however, since OreBits is a shared infrastructure actual performance may vary depending on workload from other customers.  When they become available, statistics on data performance and availability will be posted.

4.9 Refunds:

We are not able to process refunds or pro-rate any fees for any time lost due to repairs or maintenance events (planned or otherwise), nor for any datacenter-related down time. 

5.  HELP/SUPPORT REQUESTS

OreBits users may initiate support requests through the CCIT Help Desk ticketing system by emailing helpdesk@mines.edu.  While CCIT / Research Support will make every effort to respond to and resolve support requests, those support requests which require domain-specific knowledge or expertise may not be able to be handled by CCIT / Research Support alone.  In these cases, the support request may be forwarded to the POC for assistance. 

APPENDIX A:

End User Agreement

Colorado School of Mines OreBits Storage Use Agreement

 As the PI, I understand that my use of and access to the digital storage facility known as the Colorado School of Mines OreBits shall be in accordance with all of the following stipulations:

____  The PI is responsible for ensuring data stored on the OreBits Platform complies with all institutional policies as well as all state and federal laws, including copyright, the Health Insurance Portability and Accountability Act (HIPAA), the Family Educational Rights and Privacy Act (FERPA), and the International Traffic in Arms Regulations (ITAR).  Please contact CCIT Research Support for advisement of sensitive data

____  If any of the files that I store on the OreBits Platform are subject to one or more agreements with any  Institutional Review Board (IRB), including but not limited to the IRB of Mines, then I will take full responsibility for ensuring full compliance with such agreement(s). 

____  If I am collaborating with colleagues who are at institutions outside of the United States of America (that is, outside of both US states and US territories), then I will take full responsibility for ensuring that those colleagues do not access the OreBits themselves, but rather I and/or or other members of my team who are at US institutions will access the OreBits on behalf of the entire team. 

____  I understand that, if and when I cease to be employed by and/or a student at an institution in the United States of America, then access to my files on OreBits will be available only to those of my collaborators who are employed by and/or students at US institutions. 

____  If I am one of the Principal/Co-Principal investigators of a team, then I will take full responsibility for ensuring that any other members of the team are likewise in full compliance. 

____  In the event that the OreBits Platform ceases providing storage or any comparable resource, then I will take full responsibility for transferring any and all relevant files to other storage resources, and in a timely manner. 

____  I will take full responsibility for ensuring that I keep abreast of and comply with changes to any of the relevant laws, policies and circumstances described above. 

APPENDIX B:

Order Form                                                                                               

Project name:  ________________________

(should be specific to group or project; will be used in directory path; should not change over life of project; cannot include spaces.  Good example: smith_mri.  Bad example: labdata.)

Service

Cost/TB/mo

TB (whole numbers)

Months (12)*

Total Cost

Active

$9.93

     

Active with replication

$19.86

     

*Initial account setups will be prorated until June 30th to ensure concurrent subscriptions

Requested start date ______________

Project PI _______________________

  • Email ____________________
  • Dept _____________________

Project POC _____________________

  • Email ____________________
  • Dept _____________________

Usernames for access: ­­­­­­­­­­­­­­­­­­­­­­­­­­­­­


Font Size