1. Overview of AdvancedTCA, AMC, µTCA
xTCA
Overview of AdvancedTCA, AMC, µTCA
Hope my voice lasts - or maybe better if it doesn’t!
Before I begin I would like to say that AdvancedTCA was initally over-hyped - a common problem with new systems. I believe ATCA is now a real commercial product and growing rapidly. Joe Pavlat (chair of PICMG) cringed at the first year talk of a $10’s billions in a year or two. New systems don’t develop that rapidly.
AMC has come along as part of ATCA and the basic features set by that spec.
Later some of the ATCA developers decided that the AMC module could be used in a small form factor system and be “inexpensive”. Some one was rash enough to talk about the $500 dollar system - some friends of mine in companies that are interested in mTCA would like to choke him for that statement (new hype again).
This was a good idea that was not completely thought thru and has caused some indigestion in the committee. Not uncommon when you have an enthusiastic group.
That having been said I don’t want to discourage you from mTCA - it definitely has a place in systems that we will build. I just want to put it in perspective and show the evolution of the spec. and why it evolved as it did.
4 Dec 2007
DESY Crate STD. Workshop

2. General Description of xTCA
Very broad overview of ATCA - time!
AMC Modules
ATCA Carrier
µTCA
AMC
ATCA
My emphasis is on how the ATCA-AMC specifications were used to develop mTCA and describe what was necessary to port as many features as possible to mTCA. Also, describe what didn’t make it.
I think by understanding the ATCA carrier card the relation to mTCA will be clearer
4 Dec 2007
DESY Crate STD. Workshop

3. DESY Crate STD. Workshop
Advanced TCA
Telco driven specification
20 year lifetime - Telco issue
High Availability - typically 5 to 6 nine’s
Redundancy
Hot swap of everything
System management fundamental to specification
Monitor
Control
Industrial support
Telco shift from internal to external supplied hardware
Non-Telco uses of specification
Lets start with a quick look at AdvancedTCA. I presume that you already have some familiarity with this piece.
The lifetime requirements for TELCO should make us more comfortable since we also need long life systems.
SM means that one can have “swivel chair control” (Chuck Byers of Alcatrel-Lucent) - I.e. remote operation of system
The technical director of PICMG told me that they have been pleasantly surprised at the amount of non-Telco interest in ATCA.
4 Dec 2007
DESY Crate STD. Workshop

4. Status of ATCA Hardware
Typical start-up curve for new “standard”
At knee transition from R&D to production
Year
Volume
2002
2007
“Hockey Stick” Curve
~$1x109
Common curve that has existed for all standards. Hype at beginning was usual and overly optimistic. In the case of ATCA the knee has occurred at about $1B (less in €)
4 Dec 2007
DESY Crate STD. Workshop

5. DESY Crate STD. Workshop
High Availability
This is an issue for large systems that have to “stay up” - not just Telco
Research machines
Military
Access limited critical electronics
HA does not ignore Reliability but needs architectural features to implement
If an HA failure has occurred then MTTR becomes important
operating on Reliability only until repaired.
HA especially important for Control Systems
necessary to diagnose failures
The point here is that operational issues can impact HA.
4 Dec 2007
DESY Crate STD. Workshop

6. DESY Crate STD. Workshop
ATCA Features
Redundant shelf controllers - Shelf Manager
Redundant Power
Redundant links
Redundant modules
Update channels - module to module direct link
Hot swap
Mezzanine card - AMC
No busses!
Soon - JTAC on 4 unused pins in power connector
AMC also has HA features
No busses should be tempered. One can have a few optional bused or radial lines. The choices need to be evaluated as to the impact on HA.
4 Dec 2007
DESY Crate STD. Workshop

7. DESY Crate STD. Workshop
ATCA Hardware
Board Sizes
Front mm x 280 mm (8U front panel)
Rear mm x 70 mm
Pitch - n x 6HP (30.48 mm)
Power
Front W
Rear - 20 W
Temperature
Specifications at 55°C air temperature
Maybe 2x power at 20°C
Air ∆ - 10 °C shelf inlet to outlet
Power pins rated for 9 A at 10 °C ∆
If temperature was lowered could increase power - IF proper cooling design if module
Power pin ratings allow for increase in power. Some discussion in the PICMG 3. Committee to speck a power option where the temp is lower.
4 Dec 2007
DESY Crate STD. Workshop

8. DESY Crate STD. Workshop
Shelf Management
IPMC
Shelf Mngr
Fan Tray
Power Entry
IPMB (2x)
Key to ATCA specification
IP - E-net (2x)
4 Dec 2007
DESY Crate STD. Workshop

9. DESY Crate STD. Workshop
Module Management
Need to start talking about AMC’s - a bit of a mix with ATCA but there is no clean way to divide the topics.
Radial best for HA (bus vs radial choice)
IPMC - Intelligent Platform Management Controller
IPMB - Intelligent Platform Management Bus
Shelf manager communicates with host thru Ethernet
4 Dec 2007
DESY Crate STD. Workshop

10. New AMC Size & Terminology
Board Heights
Extended Full-height  Full-size - 6HP Panel
Full-height  Mid-size - 4HP Panel
Half-height  Compact - 3HP Panel
Board Width
Single-wide  Single (or assumed to be Single if not specified) - ~75 mm
Double-wide  Double - ~150 mm
Board Size
Single - ~75 mm x 180 mm
Double - ~150 mm x 180 mm
Extended Full-height was Full-height and Full-height (Mid-size) did not exist at first
Note: in double AMC’s there is no connection in what would be the second connector position!
To me the terminology has been confusing. The height seems to me to be the width, e.g. the width of the front panel. The width is the height of the front panel. We are stuck and might as well get used to it.
4 Dec 2007
DESY Crate STD. Workshop

11. DESY Crate STD. Workshop
AMC Electrical
Voltage
12 V for payload
3.3 V for management
Current
0.75 A per pin - 8 pins
Power
Power derived from Carrier Module
Single
Double
Compact
24 W
48 W
Mid-Size
30 W
60 W
Full-Size
80 W
Note that current only allows 72 W - spec under review.
All AMC power usage has to come out of the 200 W allowed for ATCA
Aside: IBM has specified AMC’s in their new server blades. Blade picks up a lot of ATCA, not plug compatible.
4 Dec 2007
DESY Crate STD. Workshop

12. AMC Port Mapping Strategy Extended Options Region
AMC Pin Assignments
MMC
GbE
SAS
Sync
Clocks
JTAG
12V PWR
3.3V MP 85 86
170
Common Options Region
TCLKB
FCLKA 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
TCLKC/D 17 18 19 20
TCLKA
AMC Port Mapping Strategy
Fat Pipes Region
MISC
Extended Options Region
PCIe GbE/10GbE
PCIe on 4-7 GbE or XAUI on 8-11
Basic Connector
RTMs
Extended Connector
Connector in two versions - 85 single side or 170 double side. Note: ONLY one connector per AMC - in double width the “other” connector does not exist.
MMC - Module Management Controller
SAS - Serial Attached SCSI
XAUI - X = 10gig, AUI - Attachment Unit Interface
Note that JTAG has existed but no standard way to connect it to the system
Clocks are for Telco but no reason not to use them for research, e.g.synch to beam
4 Dec 2007
DESY Crate STD. Workshop

13. DESY Crate STD. Workshop
AdvancedTCA Carrier
ATCA module with 6HP panel
Standard ATCA power
Standard ATCA management
Specified for :
8 Compact Single AMC’s
4 Mid-size Single AMC’s
2 Full-size Double AMC’s
Many combinations between all Compact and all Full-size Double
12 and 3.3 Volt converter for AMC power
IPMC controls IPMB-L for control of AMC
Fabric switch connects AMC’s to Zone 2
Extended options can connect to Zone 3
4 Dec 2007
DESY Crate STD. Workshop

14. ATCA Carrier Card AMC Links
Zone 1
Switch
Zone 2
This drawing only addresses links. Note that as many as 8 AMC’s could be here
Zone 3
AMC
4 Dec 2007
DESY Crate STD. Workshop

15. ATCA Carrier Power for AMC’s
12V
Current Limit
AMC
48V
Power
Zone 1
MMC
3.3V
Zone 2
Module Management Controller
Zone 3
Current Limit
AMC
MMC
4 Dec 2007
DESY Crate STD. Workshop

16. ATCA Carrier AMC Management
Zone 1
IPMC I
MMC
IPMB
Zone 2
IPMB-L
MMC = Module Management Controller
I = Isolater
Could also be radial with protection in IPMC
Zone 3
AMC I
MMC
4 Dec 2007
DESY Crate STD. Workshop

17. Status of µTCA Hardware
Startup for µTCA if growth is “normal”
“Hockey Stick” Curve
Volume
Where is mTCA in development? Much hype but cooler heads think the above is most realistic.
Knee occurs usually at 5 years. Might be a bit sooner since much has been proven in the ATCA/AMC world already
knee
2006
2010/11?
Year
4 Dec 2007
DESY Crate STD. Workshop

18. DESY Crate STD. Workshop
µTCA Hardware
Shelf for AMC modules
Single Tier* * = more detail later
Two Tier Mixed Width
Two Tier Fixed Single Width
Back-to-Back
Cube*
Pico
Electrical
Signals per AMC
Radial power V and 12 V
Will show in more detail later
Not enough time to do all but these two extremes should let one evaluate the other options.
4 Dec 2007
DESY Crate STD. Workshop

19. DESY Crate STD. Workshop
ATCA relation to µTCA
The next slides take the ATCA specification and relate it to the hardware needed in µTCA
The ATCA carrier module is the basis to relate the requirements set in ATCA into the µTCA world.
First the AMC Module Management functions
Next the AMC connectivity as is it implemented in µTCA
Carrier power functions for AMC’s that has to be translated to µTCA for hot swap
Shelf Management hardware
4 Dec 2007
DESY Crate STD. Workshop

20. µTCA Module Management
SM/IPMC Typically in Hub
4 Dec 2007
DESY Crate STD. Workshop

21. µTCA Carrier Card AMC Connectivity
Zone 1
Switch
Zone 2
µTCA Hub Module
Zone 3
AMC
4 Dec 2007
DESY Crate STD. Workshop

22. Radial Power Connections
µTCA Carrier Power
12V
Current Limit
AMC
48V
Power
Zone 1
MMC
Input Power
DC or AC
µTCA Backplane
3.3V
Zone 2
Radial Power Connections
µTCA Power Module
Zone 3
Current Limit
AMC
MMC
4 Dec 2007
DESY Crate STD. Workshop

23. DESY Crate STD. Workshop
µTCA Shelf Manager
Up to 16 µTCA Carriers per Shelf Manager
Shelf Manager is a logical function
Interface between Carrier Manager and SM is logical, typically IP based
Implemented on any FRU
MMC’s (Module Management Controller) in all AMC Modules
MCMC’s (µTCA Carrier Management Controller) in µTCA Carrier Hub (MCH)
EMMC’s (Enhanced Module Management Controller) in power supplies and cooling units.
FRU - Field Replaceable Unit
4 Dec 2007
DESY Crate STD. Workshop

24. Management of µTCA Shelf
2 max.
4 max.
System Manager
Carrier FRU Info Device
EMMC
Cooling Unit
EMMC
Power Module
I2C
IPMB-0
IPMB-L
PS1#(1:16)
Enable (1:16)
MCMC
MCMC
AMC1
MMC
AMC12
MMC
Carrier Manager
MCH
MCH
Shelf Manager IP
4 Dec 2007
DESY Crate STD. Workshop

25. µTCA Hardware Examples
Backplanes
Shelves
Industrial
Military
System Modules
Power Module
Hub Module
Cube
Alternate “µTCA” shelf
4 Dec 2007
DESY Crate STD. Workshop

26. DESY Crate STD. Workshop
ELMA µTCA Backplane
16 slot backplane, 26 layers
Dual star
10 AMC
2 PM
2 MCH
Will look at Hub and Power Module in next slide.
4 Dec 2007
DESY Crate STD. Workshop

27. CorEdge Networks Hub & Power Module
Note connectors on Hub
Power feed from front
4 Dec 2007
DESY Crate STD. Workshop

28. DESY Crate STD. Workshop
ELMA µTCA Shelf
Uses previous backplane or others
5U chassis
Front air intake - rear exhaust
4 Dec 2007
DESY Crate STD. Workshop

29. Schroff µTCA Shelf (Cube)
Front view
4 Dec 2007
DESY Crate STD. Workshop

30. Schroff µTCA Shelf (Cube)
Rear view with power unit open
Note that logic has been placed on the backplane - maybe not good but does save money?
4 Dec 2007
DESY Crate STD. Workshop

31. Schroff 5 Slot Cube Specs
MicroTCA Backplane with radial IPMI from MCH in Slot1# to all AMC slots, supporting:
1 MCH Single Full-size slot
1 AMC Single Full-size slot for a CPU Board
2 AMC Single Full-size slot for HDD or PCIe Boards
1 AMC Single Full-size slot for a HDD or CPU or PCIe Board
Onboard power management for all AMC slots
Active cooling through a 12 V Fan (170 m3/h (100 CFM)) for the MicroTCA Boards and a 12 V Fan (13,5 m3/h (8 CFM)) for the power supply
Integrated 150 W AC Power Supply with wide range input
4 Dec 2007
DESY Crate STD. Workshop

32. Hybricon Military ATR Box
Work on-going. Hybricon is a major player here.
4 Dec 2007
DESY Crate STD. Workshop

33. Hybricon ATR Spec Sheet
AC/DC front end power supply supporting 110/220VAC operation with 48VDC output to DC MicroTCA Power Module.
Shock Mount MicroTCA card cage inside the outer enclosure.
150MM Front Row 3 Full Size Double Width (150MM) slots for 150MM AMC Modules 2 Full Size Double Width slots with conversion module to convert to (2) Single Width slots
75MM Rear row 2 Full Size Single Width (75MM) Slots for 1 DC or AC MicroTCA Power Module (1) Compact Single Width (75MM) AMC slot (accessible only with DC power supply)
1 Full Size Single Width (75MM) slot for MicroTCA carrier Hub (MCH)
4 Compact Single Width (75MM) AMC Slots
Designed to cool 80W per double width full height slot and 40W per single width full height slot at 10,000 feet altitude
Simulated 36 CFM average flow rate per full size slot tools 80W per double width (150MM) full size slot, in addition to 40W per single width (75MM) full size slot, and 20W per single width (75MM) compact slot at 10,000 feet altitude.
Pressurizing commercial grade fans mounted to card cage at front panel with honeycomb air inlet panel and application specific I/O connectors
900W Embedded AC/DC Power Supply: MIL-STD-461 Line Voltage Filter and EMI Containment.
Data sheet for previous slide
4 Dec 2007
DESY Crate STD. Workshop

34. DESY Crate STD. Workshop
µTCA in AdvancedTCA
Horizontal ATCA Shelf
Use ATCA Carrier Modules
Get rear Zone 3 access for AMC’s
Can use RTM
Shelf Manager can be used
3 Slot Shelf
2 Carriers with 8 compact single AMC slots each (16 total)
1 network module
“Simple” backplane
This might be an alternate to mTCA. Uses ATCA in a mTCA way. Should be cost effective and give one the rear I/O.
4 Dec 2007
DESY Crate STD. Workshop

35. DESY Crate STD. Workshop
µTCA in AdvancedTCA
2 Slot Horizontal Shelf
Front
Sorry about the number of slots but this gives the general idea.
Rear
4 Dec 2007
DESY Crate STD. Workshop

36. Standards vs Specifications
xTCA Documents are Specifications, not Standards
Specifications are sometimes used by industry because of the bureaucracy of the governing bodies - time-to-market
Standards are governed by organizations such as ANSI, IEEE, IEC, etc.
Rules on changing standards
Review every 5 years
Gives stability
Specifications are governed typically by industrial groups
Rules on change are set by group writing specification
Review are up to group
Stability
Not an issue with a large industrial group that has long view
Could be an issue with small group without long market view
xTCA will be stable because of the constituency of the governing group
Legalism’s
4 Dec 2007
DESY Crate STD. Workshop

37. DESY Crate STD. Workshop
xTCA Related Groups
PICMG
The starting point for xTCA - they “did it”
SAF - Service Availability Forum
Formed by some ATCA developers and others
High Availability and management software interface specifications
Scope Alliance
ATCA specifications are very broad with many options
Define a common subset to promote interoperability
Recognized that there were choices being made by most suppliers
Documents”
AdvancedTCA HW Profile, v May 2007
Linux Profiles, latest v Feb 2007
AMC Port Map Gap-Analysis, v1.2, 20May 2007
Others
PICMG - good place to search for products. Click on “Product Listings”. Can also get short form specs (if not a member) from “Specifications”
4 Dec 2007
DESY Crate STD. Workshop

38. Physics “SCOPE” Document
Based on two SCOPE Alliance documents
HW Profile
AMC Port Gap-Analysis
Intent
Engage research community in a discussion on a common set of guidelines for building xTCA hardware
Try to optimize the exchange of designs, hardware, etc.
Content of Document
Minimize changes from SCOPE document and track them
Add items such and hardware for RTM’s and Zone 3 connectors
Propose alternate rack and shelf cooling systems - fluid based
Further restrict network protocols to PCI-e and E-net
Needs: software section to be expanded on.
4 Dec 2007
DESY Crate STD. Workshop

39. ATCA for Physics Collaboration
Proposed Organization
Ray Larsen distributed this recently for comment
4 Dec 2007
DESY Crate STD. Workshop

40. DESY Crate STD. Workshop
Request
Please look at “straw man” document
Give feed back so a revision 2 can be produced
Information on prototypes is of use to the document writers
Suggest other topics that could be included
Currently it is light on software, too early?
Should software be in another document?
More information on analog design experience
4 Dec 2007
DESY Crate STD. Workshop

41. DESY Crate STD. Workshop
Conclusions
xTCA family offers an excellent next generation platform to build on
Industrial support ramping up
Long lifetime because of Telco support
Telco will be first to drive high volumes, hence lower costs
Lots of interest from other sectors of the modular electronic market - more non-Telco hardware
Software support for xTCA - something new for new modular electronics in our business - VERY GOOD!
Caution, some potential incompatibilities exist in taking AMC’s to µTCA
4 Dec 2007
DESY Crate STD. Workshop

42. DESY Crate STD. Workshop
Thank You
Robert Downing
SLAC Consultant
Maybe mention RoHS? BGA’s appear to be a real problem. This is a big reliability issue. Consumer part of industry doesn’t care since they like rapid replacement.
4 Dec 2007
DESY Crate STD. Workshop