User Program - While at the Beamline - Data Management and Backup
Supported media
GM/CA @ APS recommends and supports external firewire (IEEE 1394),
USB3, USB2 and ESATA disks as
inexpensive and fast way to backup users data. The table below provides a speed
comparison of different connector types for typical 5400rpm HDD external drives
(much faster SSD drives are not included since they are uncommon):
| Connector | Bus Speed | Real Speed | 200GB backup time | Restricted by |
|---|
| USB3 | 500 MB/s | 40-50 MB/s | 1 hour | disk spindle |
|---|
| ESATA | 300 MB/s | 40-50 MB/s | 1 hour | disk spindle |
|---|
| Firewire | 50 MB/s | 24 MB/s | 2.5 hours | bus |
|---|
| USB2 | 40 MB/s | 20 MB/s | 3 hours | bus |
|---|
| USB1 | 1 MB/s | 1 MB/s | 56 hours | bus |
|---|
At present not all computers have USB-3 and ESATA ports. Please
ask your host if in trouble finding the right hub.
Support for DVD disks is not provided because at the rate of 200GB of data
per experiment one would have to burn more than 45 disks.
Supported disk formats
The supported formats on external disks are FAT32,
NTFS, ext3 and HFS+.
We have only Linux computers at the beamlines. Therefore some Windows-specific
and MAC OS specific extensions, for example encrypted drives like WD Passport
that require some Windows or MAC OS utility to unlock the encrypted partition,
may not work. When choosing format for your hard drive, keep in mind that FAT32
has the best record in terms of portability between different operating systems
(Windows, Linux and MAC OS), while HFS+ has the shortest history of support
under Linux. Please, try to avoid disks with multiple partitions and bringing
disks with data as in some problematic cases your disk may need to be re-partitioned
or reformatted. The HFS+ format is supported on the second-day workstations ws4
and ws7 only.
Firewire and USB2 disks can be connected to any beamline workstation except
for ws2, ws3 and mar. In the second-day area ESATA disks can be connected to
ws4 and ws5 and USB-3 disks to ws4. Usually disks are mounted automatically
upon connecting provided user has logged in before connecting the disk. If the
disk is not auto-mounted, go to Applications -> System Tools > Disk Utility,
then find your disk in the left-pane list, choose the right partition and click
the "Mount Volume" button:
Additional details of mounting external disks under Linux operating system
are described elsewhere.
Network transfers
GM/CA @ APS runs 10Gb network and users are welcome to SFTP or FTP out
their data to the servers at their home institutions. External access the
GM/CA SFTP and GridFTP servers is
also possible, but needs to be requested in advance. The access can be allocated
for the time of experiment plus one day. That policy is enforced in order to
avoid conflict of resources between different user groups. The real rate of
remote data transfer will depend on the network connectivity to the users
institution. Please check the Network Speed
Testing page. Unless there is a bottleneck on user's side, one can expect
the SFTP rates of 5-8MB/s and GridFTP
rates of 20-25MB/s. For a 200GB data set it may take about 10 and 3 hours
respectively.
Additional Information
For the latest changes with the backup procedures and detailed technical
instructions please refer the Backup Manual
(PDF).
| 
