Communications infrastructure for a teraflop computer
Building on the successful ACPMAPS technology, we
propose to build a prototype switching network that would lay
the groundwork for a next generation ACPMAPS supercomputer.
The project can be done now with a minimum budget (~$50,000) and
minimal effort (2 FTE for 1 year) using FPGAs and commercial switching
chips. The following table lists the prototype specifications
in comparison to the final target specifications and the existing
|Target System||Prototype||Current ACPMAPS|
|Cycle Time:||2 to 10 ns||20 ns||200 ns|
|Word Width:||64 bits||32 bits||32 bits|
|Bandwidth Per Port:||800 to 4000 Mbytes/Sec||200 Mbytes/Sec||20 Mbytes/Sec|
|Number of ports:||64||20 or 40||16|
|Total Bandwidth:||51.2 to 256 Gbytes/Sec||4 or 8 Gbytes/Sec||.32 Gbytes/Sec|
|Connection Latency:||40 ns||80 ns||1200 ns|
|Software user-to-user connection latency:||300 ns||500 ns||30,000 ns|
The prototype switch will consist of the following PC-boards:
Switch Core: Parts $200 Fabrication $1,500 Design effort:1 week Layout: 1 week
Contains an 8-bit slice of the 40-port switch. Five of these will be required for to implement a 40-bit data+control path.
Consists of 1 or 2 I-Cube® switch chips and connectors.
Switch Arbiter: Parts $200 Fabrication $1,500 Design effort 2 Months. Layout: 1-2 weeks.
Services requests from 64 slots and configures switch core.
Consists of a small FPGA and connectors.
CPU card: Parts $2,000 Fabrication $1,500 Design effort4-6 Months Layout: 1 Month.
This is a mock CPU card that interfaces to the switch. Several of these will be required for a system test.
Consists of a port-arbiter chip (FPGA), DMA Queue processor Chip, RAM, and a cheap microprocessor with memory.
Mechanical: $10,000 4-6 Months
Substantial effort needs to be spent on mechanical design. This includes design of the cylindrical crate and card guides, cooling, and connector design. The prototype can be built using off-the shelf connectors, but special connectors may be needed to achieve the bandwidth and reliability required by the target system.
System Design: 1 Month
Some time should be spent in meetings with ACPMAPs alumni discussing architecture, hardware and software protocols.
Software: 4 Months
A minimal amount of software needs to be written to do basic functionality and performance tests. This includes porting of a small operating system and writing a hardware interface. It also includes high level software that will simulate the communication requirements of a user-level program.
Conversion from prototype to target:
Several major design efforts are required to convert the prototype system to the target system. Work should be done now to specify these efforts. Major efforts include:
Conversion of FPGAs to higher speed gate arrays.
Development of a switch chip and electrical protocol that can handle sub 10ns cycle times.
Develpment of a crate-to-crate link that can handle up to 4 gigabytes/Second.
Choosing a high speed microprocessor and putting several on a board with large amounts of RAM.
Developing the operating software and user libraries.
It will take approximately 5 full time engineers two years to convert the prototype into the target system.
Details of switch core
Back to ESE home page