Fermilab Shutdown Report
8/14/09 – 8/21/09
LINAC
Shutdown Week 10 – On Schedule
- Activities
- 400 MeV beam to the dump on 8/17/09
- Preaccelerator
- H - is operational putting out 55.5 mA
- Repaired a vacuum leak on the I - column
- Reinstalled the electrostatic chopper in the 750 line
- Flushed H - water resistor
- Low Energy
- Water leak in LRF3 modulator (happened twice this week!!)
- Replaced 1 out of 3 Switchtubes in the modulator
- Replaced water fittings/hoses for the supply/return line
- Tune up stations for better performance
- Full calibration of the LLRF parameters on all stations
- High Energy
- Cleaned strainer in KRF6 Solenoid A manifold
- Installed vibration damping hardware on the Debuncher water skid electronic rack
- Repair pressure transducer on the distribution skid
- Ongoing effort on debugging the new VXI software
- Next Week
- Support NTF patient treatment (Tuesday, Thursday, Friday)
- Address problems as they reveal.
- Provide beam to Booster commissioning.
- Continue machine tuning
Booster
Shutdown Week 10 – On Schedule
- Tunnel Activities
- Completed GMPS installation Friday and Saturday. Turned on all 4 supplies on Saturday and ran through weekend and this week
- Completed Corrector checkout and PS turn on over the weekend to establish a list of problem supplies for Monday follow-up
- HLRF was ready for beam but weak PAs still need to be replaced next week. Limited spares
- Booster was ready for a beam permit sign-off on Wednesday. Permits signed end of day shift
- Established Circulating beam Wed. 8/19 evening by ~1730hrs and to 8GeV dump ~2250hrs
- Today’s plan
- Currently running on low intensity beam permit.
- Continue Orbit tune-up
- Weekend plan:
- Continue beam tune-up/studies
- Next Week’s plan
- Conduct equipment repairs as needed
- Limited access time (controlled access)
- Coordinate Booster access with MI8, MTA/Linac access (Wednesday possibly)
- HLRF Repairs, PA and Air Filter replacements.
- Misc jobs.
- Tune up beam at higher (typical) intensities.
- Dedicated beam studies
Antiproton Source
Accumulator 4-8GHz DP Kicker Upgrade: Complete
All water and electrical connections complete and power testing is being done today. There are some controls problems with the new TWTs being addressed. We did have one of our new ion pumps ground faulted after the bake was done, but thanks to the LINAC vacuum techs, we cleared the fault by zapping the pump.
Debuncher Septum Magnet Replacement: Complete
The magnet was pulse tested for several days but the vacuum was a couple of decades worse than before the shutdown started. It also gets worse while being pulsed. We are doing a low temperature bake (~105C) to try and dry out the magnet. All indications right now are the bake is going to help quite a bit.
Other Work
- RF system testing
- LCW leak repairs are almost complete. System leak rates are very low at ~15 gallons/day
- Target station work continues. A vacuum leak in the target SEM will prevent installation during this shutdown, but will be ready later in September. Inspection photos of the target and lens were taken
- Vacuum system controls problems addressed (CIA crate troubles, vacuum valve status)
- Smoked a card and the backplane of a stepper motor crate. Repairs in progress.
- AP0 shielding blocks are in place and will need to be locked.
- We’ll be ready to start safety system tests on Monday 8/24
- When safety system testing is done, secure and start power supply testing.
- We will stay in controlled access mode after Monday, 8/24
- Begin with reverse protons to measure orbits/apertures as soon as beam becomes available, start stacking ASAP
Main Injector
Tunnel Work
MI-8
- Baking multiwire at 840
- Need ~6 hours to end bake and pull equipment. (No Booster beam)
MI-40
- Lambertson bake continues until Tuesday Aug 25 th
- All seven kickers are in the tunnel!
- Abort kicker cable tray relocated
MI eCloud (MI-52)
- Installation complete
- Last cables being connected
Other Work
- MI-30 collimator LVDT investigation continues
- MI-30 collimator mask installation completed last night
- MIPS & MIRF work going well, on schedule
- LCW is on and conditioning
- MI-52, MI-62 and RF systems have been filled
- Power supplies are being valved in
- LCW looks good at 7+ Meg Ohms
Surprise work
- MI-30 vacuum went bad Tuesday morning
- Crews have replaced 3 ion pumps
- Leak check points to corrosion on an elliptical bellows downstream of Q308
- Working with Safety to plan ALARA
Next week
- KRSS power outage for MI/RR Monday morning 7:00 – 7:30 am
- Access MI-8 to turn on ion pumps after the outage.
- Turn on all pond pumps
- Finish bakes for 840 wires (Booster beam off) and MI-40 Lambertson
- Finish leak checks and button up vacuum at MI-40 kickers
- Complete any remaining MI-30 collimator mask/LVDTs work
- Alignment, alignment and more alignment
- Re-Key Tuesday night: Wed, Thurs, Friday must get tunnel access key from Mary (staying supervised access except for safety tests).
- Safety tests Wednesday evening – secure MI/RR starting 4:00 PM (???)
Recycler
Recycler
- RR129 & RR400 repair/modifications (99% Complete)
- Power supplies maintenance ( Complete)
- HLS installation at MI-10 ( Complete)
- May be one more round of ‘air burping’ next week
- Vacuum maintenance
- Looks good at 401-409
- However, can see effect of the MI Lambertson bake on the vacuum pressure
- Bake of IPs in the 800 line started
- Likely terminate middle of next week
- Started pump down at 103-109 on Monday
- Very similar to 401-409 so far (which is good)
- Move 6 magnets ( complete)
- Cable tray was repositioned (2 nd time) at MI-40 to allow alignment surveys without changing fiducials
- Back to its original location where a second tray won’t be needed
- Lowered to ~water pipe level elsewhere
- Survey of the area was performed on Monday (following the cable tray reconfiguration)
- Aligned magnets back to before the shutdown ‘as-found’
- That does not take into account the level runs (i.e. it is a local alignment)
- 1-2 GHz stochastic cooling system converted back to longitudinal
- CIA crate activities ( Complete)
Electron Cooling
- Pelletron maintenance ( Complete) but…
- Installed and tested modified modulator (last Friday)
- Cleaned on Monday
- Obtained running permit and transferred gas on Wednesday
- Turn on and preliminary checks (before conditioning) on Thursday
- Transfer gas today for repairs!
- ‘Group 3’ electronics problem
- Likely failure of a 15 V PS
- Unfortunately, not a ‘new’ problem
- Other issue to address
- Water/Glycol system cleanup and upgrade ( Complete)
- NMR probes replacement ( Complete)
Tevatron
Other Items
- Replacement of bad stands finished
- All magnets un rolled
- TeV RF water leaks fixed
- Replaced D 11-3 re aligned D11-2 to 11-5
- D-1 ready for Hi Pot
- Vertical crystal collimator installed still needs controls cables
- QBS at F46 back in wall
- Need to finish HLS at B0
- Bad O ring at C48 caused cryostat vacuum to go back simple fix no need to warm house
- Bad O Ring at B3 caused loss of insulating vacuum this has been fixed
NuMI
Shutdown Week 10
- Complete Survey on Target NT-03
- Complete Final Adjustments of Target
- Complete Final Survey
- Install water lines & start up RAW Skid
- Remove Rail Car System from Target Hall
- Remove Cameras and Controls from Work Cell
- Complete Flushing of Target & Horn RAW Skids
- Start Installation of Antenna Repeater for TH Crane
- Dry Out Pre Target Drains and Reset Water Sensors
- Connect Pre Target Drain Pumps to Emergency Power Panel
- Profile Monitor at PM121
- Replaced Stand & Profile Monitor
- Complete Survey
- EPB Dipoles PEEK Installation Continues
- Installed water lines and shielding HV101-4, 101-5 & HV101-3 (Final survey next week)
- Flush & Start Up Absorber Ground Water Cooling Loop and put Filter System in Line
- Continue Installation of AH Dehumidification System
- Complete installation of controls system
- Start up Seresco cooling coils
- Start up Munters dehumidifiers
- Complete test of condensate pump system
- Complete Installation of Hadron Monitor
- Install cable tray and wire raceway on Absorber
- Complete rewiring of Hadron Monitor
- Complete Hadron Monitor Testing
- Reinstall Electrical Removed for HM Replacement
- Lighting, Fire Protection, Interlocks
- Install Muon Monitor Absorber Curtains
- Install Interlocks on New Wall & Door to Emergency Walkway in Absorber
- JASMIN Group Installed Activation Samples on Downstream face of Absorber
- Complete Anti Neutrino Mode Horn Pulse Testing & Thermal Imaging (Monday PM)
- Temporary Core Change, Search & Secure, Power On Access
- Complete Belt Replacement on both Target Pile Munter Dehumidifiers
- Take Decay Pipe Gas Samples in Absorber Hall
- Complete Neutrino Mode Horn Pulse Testing & Thermal Imaging (Thursday)
- Change Cores on Thursday next week
- Install Target Hall Shielding Plug & Air Dam
- Replace & Seal R-Block on Down Stream End of Target Pile
- Power Supply Testing
- Commission Absorber Hall Dehumidification System
Cryogenic System
TeV at LN2 Temperatures
Tevatron Operations and Maintenance Status
- Continued support AD/MSD for vacuum repairs: D3, E3, F1, and Pbar Accumulator tanks are back at LN2 temperature. A1 and A2 being restored for cooldown after signoffs. A3 is ready for cooldown but awaiting A2 completion. D1 and E1 under LOTO for repairs by MSD. Concerned about magnet cryostat vacuum in B3 upstream
- LN2 inventory control and controlling magnet temperatures at LN2
- Expanders group: Expander u-tubes have been re-stung back into service
- Enormous backlog of shop repair and overhaul work
- Electrical group: Continuing upgrade of Cutler Hummer 120V power. Continuing maintenance of cryo I/O crate, electrical equipment cabinets, various low voltage electrical/electronic maintenance
- Compressor group: Mycom Compr relief valves replaced (7/8). 5/5 IHI cold old compressor have been overhauled
- Ongoing numerous FRIG/Compressor controls/instrumentation maintenance/repairs
- Vacuum pumping of various cryo equipment in FRIG bldgs to start next week
---------------------------------------------------------------------------------------
Pending RT houses and relief maintenance, Tevatron cryo is on schedule with purification starting next week.
CHL HELIUM and NITROGEN PLANTS
- 99.9% of all scheduled routine mechanical and electrical maintenance is complete
Current operations:
- Nitrogen Reliquefier TSI 2.5MW compressor was successfully tested after bearing/seal overhaul by outside subcontractor. Plant cooldown possibly next week if ICW work (fire protection valve packing) is completed by FESS
- Repaired turboexpanders for the CHL coldbox-2 were delivered and installed; test run either today or Monday after completion of system purification, which is currently in progress. Backup plan is to restart with coldbox-1 that operated before shutdown
- Compr “B” purified and currently purifying CB-1 and CB-2 concurrently.
- Mobile Helium Purifier at CHL has been set up for Tevatron FRIG purification
-----------------------------------------------------------------------------------------------------
We're back in full 24/7 shift operations, supporting Tevatron FRIG system monitoring and operations, as well as A0 Test Caves and such.
Controls
Last Week and this Week
- Finding & Fixing Bugs
- Startup Support
- General Control System Maintenance
- MAC room cooling
MiniBooNE
Sump pump problem fixed.
Collider Detector at Fermilab
- Started flowing gas to the detector
- Began and continuing to slowly power up the solenoid
EE Support
- MI power supply maintenance complete
- MI-20 lower transformer oil leak repair has been planned
- MI-20 quadrupole magnet LCW leak was repaired
MTA
Their new magnet will possibly be installed on 8/26/09.
FESS
Electrical and High Voltage
- Kautz Rd SS – work completed on SF6 breakers. Will complete work early next week on compad & will not interfere with the Mon [8/24] power outage
- Most findings were mice related – will install more bait boxes
- MI – fdr PMs work to complete at MI-10 & 40 due to weather/construction rescheduling
- Lighting – corrective work orders being scheduled as necessary for ballasts, etc.
Mechanical
- MI-60 - glycol line leak repaired
- MINOS – glycol under ground chilled water line repaired
- CDF – supply air handlers work scheduled – sheave belt alignment & dirty coils
HVAC
- MI – dehumidifier checks completed. Marvair AC checks scheduled for week of 8/31
ICW
- Meson Line valve [14”] – replacement WIP includes 2 additional shut off valves.
- Spot flushing – at Meson line & strainer checks at switch yard/MuCool/CUB on Tues [8/25]
CUB
- •55LCW chiller #3 - tube & header replacement completion /start up date re-set for tentatively 9/14 due to procurement hold up & contractor [Trane] schedule revisions
- Comfort chiller #5 – required more extensive work/preparation [refrigerant clean-up & testing] with a start up date mid next week
Switchyard 120 – Work has not started
D-Zero – On Schedule
Mechanical – On Schedule
- Will be out of MI by 8/28/09
Instrumentation– On Schedule
- Finished the termination of the eCloud cabling
- Dealing with the laser profile monitor
Alignment
Alignment
- Align 7 new kicker magnets to be installed in the MI40 area. Working on today
- As-found all the Main Injector and Recycler components in cells102-109. The essential components are: Recycler = magnets and cooling tanks no trims. 40 Components
- As-found all the Main Injector and Recycler components after construction back fill in cells 341-404. Align all kickers and new beam pipe. 40 Components
- Realign 6 EPB dipoles @ (HV101 magnet string). Rough Align Completed
- Target Alignment Today
- Stickmic survey of Muon system including CF and EFs after closing detector. The start and finish dates are preliminary and depend on shutdown progress.
- Align 2 EPB dipoles in enclosure B (VH94) before reworked EPB dipoles. Align 2 new dipoles
- Align new permanent magnet to be installed on the Linac side.
- Replace/ Install SEM AP0
Safety
Safety
- Shutdown ALARA plans continue to be very close to as planned
- Accelerator restart sheets are available from Mike Gerardi
Interlocks
- Audio warning system audio level control installation
- Misc. Tevatron Interlock repairs & ground fault diagnostics installation
- MINOS Absorber interlock modifications
- Safety System Testing
- Interlocks tested in past week: MI/TeV Crossover, Enclosure B, Switchyard, MI-12A&B
- Interlock testing scheduled for next week: Pbar (all enclosures) and Main Injector
Shutdown Safety
- Two recordable injuries (forehead laceration and complications from a wasp sting) in the past week
Continue using the Take Five principles before, during, and after work activities to:
|
