Communications infrastructure for a teraflop computer

Building on the successful ACPMAPS technology, we propose to build a prototype switching network that would lay the groundwork for a next generation ACPMAPS supercomputer. The project can be done now with a minimum budget (~$50,000) and minimal effort (2 FTE for 1 year) using FPGAs and commercial switching chips. The following table lists the prototype specifications in comparison to the final target specifications and the existing ACPMAPS system:

	Target System	Prototype	Current ACPMAPS
Cycle Time:	2 to 10 ns	20 ns	200 ns
Word Width:	64 bits	32 bits	32 bits
Bandwidth Per Port:	800 to 4000 Mbytes/Sec	200 Mbytes/Sec	20 Mbytes/Sec
Number of ports:	64	20 or 40	16
Total Bandwidth:	51.2 to 256 Gbytes/Sec	4 or 8 Gbytes/Sec	.32 Gbytes/Sec
Connection Latency:	40 ns	80 ns	1200 ns
Software user-to-user connection latency:	300 ns	500 ns	30,000 ns

The prototype switch will consist of the following PC-boards:

Switch Core: Parts $200 Fabrication $1,500 Design effort:1 week Layout: 1 week
Contains an 8-bit slice of the 40-port switch. Five of these will be required for to implement a 40-bit data+control path.
Consists of 1 or 2 I-Cube® switch chips and connectors.

Switch Arbiter: Parts $200 Fabrication $1,500 Design effort 2 Months. Layout: 1-2 weeks.
Services requests from 64 slots and configures switch core.
Consists of a small FPGA and connectors.

CPU card: Parts $2,000 Fabrication $1,500 Design effort4-6 Months Layout: 1 Month.
This is a mock CPU card that interfaces to the switch. Several of these will be required for a system test.
Consists of a port-arbiter chip (FPGA), DMA Queue processor Chip, RAM, and a cheap microprocessor with memory.

Additional efforts:

Mechanical: $10,000 4-6 Months
Substantial effort needs to be spent on mechanical design. This includes design of the cylindrical crate and card guides, cooling, and connector design. The prototype can be built using off-the shelf connectors, but special connectors may be needed to achieve the bandwidth and reliability required by the target system.

System Design: 1 Month
Some time should be spent in meetings with ACPMAPs alumni discussing architecture, hardware and software protocols.

Software: 4 Months
A minimal amount of software needs to be written to do basic functionality and performance tests. This includes porting of a small operating system and writing a hardware interface. It also includes high level software that will simulate the communication requirements of a user-level program.

Conversion from prototype to target:
Several major design efforts are required to convert the prototype system to the target system. Work should be done now to specify these efforts. Major efforts include:

Conversion of FPGAs to higher speed gate arrays.
Development of a switch chip and electrical protocol that can handle sub 10ns cycle times.
Develpment of a crate-to-crate link that can handle up to 4 gigabytes/Second.
Choosing a high speed microprocessor and putting several on a board with large amounts of RAM.
Developing the operating software and user libraries.

It will take approximately 5 full time engineers two years to convert the prototype into the target system.

System Overview
Details of switch core
Communication processor

Back to ESE home page