Calculation system

In the FCSCL exists thre calculation systems: MPI cluster, shared memory systems cluster and GPU´s cluster.

MPI Cluster

The cluster for massive parallel processing is composed of 304 nodes, of which 288 are for production and 16 for development and testing. This is an extremely dense cluster format, because all cluster's nodes are HP BLx220c servers over C7000 chassis. This model is the blade “double sheet”, in other words, 32 servers per chassis. With four chassis per rack, each MPI cluster rack has a total of 128 servers and 1024 cores. As it has been reflected in the infrastructure section, this represents a full charge consumption of 44 Kw per rack.

The cluster is formed of a total of 10 chassis (one only has 16 servers). Each node of the chassis has the following features:

• Two Intel Xeon E5450 processors.
• 16 GB/RAM (ratio of 2GB/core).
• Two GbE interfaces.
• DDR Infiniband interface.

Each chassis has four onboard GbE switches and four switches Infiniband DDR. Each switch has a total of 20 ports, 16 internal through the blades and 4 external for uplink. In the case of Infiniband switches are connected to the backbone with a 2:1 ratio.

The E5450 processor is a good commitment between performance and consumption, through whom the cluster reaches a computing power of 30TeraFLOPS (RPeak) and 25TeraFLOPS in executing a Linpack test with a consumption lower than 100kw/h. It is a great data showing the highest energetic efficiency achieved, because the ratio MFLOPS/W will enable the MPI cluster Calendula be at the top of Green500 list.

Shared Memory Cluster

For those programs very intensive memory consumption, the cluster of shared memory systems is an ideal solution. It consists of 12 HP Prolinat DL 580 servers with four Intel Xeon MP X7350 processors of four cores (16 cores per node). The features of each node are:

• Four Intel Xeon X7350 processors
• RAM Memory: 8 nodes with 256GB (ratio 16GB/core) and 4 with 128GB (ratio 16GB/core).
• Two GbE interfaces
• Two dual-channel FC controllers 4GB/seg each one.
• DDR Infiniband interface
• Four discs sas 2.5” 143 GB 10k rpm (Raid5)

GPU´S Cluster

The last equipment of calculation has been added the GPU´s cluster. The GPU's systems are increasingly's demand, because its performance is excellent. In contrast, it is the difficulty of programming. But many applications of calculation can take advantages of the features of the GPU´s to get very significant performance increases.

The GPU´s cluster is composed of six HP Proliant DL160G6 servers connected to 3 Nvidia Tesla S1070 servers.

Internal Interconnection Networks

In the FCSCL exists three nets, and all servers are connected to them:

• Management Network: This network is not accessible by the user, and its purpose is to control hardware, like the management of the deployment. Each server has one or two interfaces connected to the management network. If you have an specific interface for hardware management (iLO, IPMI or similar), it is connected to this network. Moreover, all servers have gigabit interface connected to the management network, which is performed monitoring, configuration software, etc. The backbone of this network is 10GbE.
• Production network: this network is dedicated to production, and is normally used by the user. The backbone of this network is 10 GbE, and all servers have at least one GbE interface connected to it. Obviously, in this network there are separate VLANs for different environments.
• Infiniband Network: the core of intensive calculation is the Infiniband network. It is a Voltaire ISR 2012 switch, the switches that connect the blades (four for each chassis C70000), and Voltaire ISR 9024D switches  for systems that are not on board chassis blades (cluster nodes in the cluster shared memory and GPU´s cluster). Each server has at least one interface Infiniband DDR (20Gb/s).

Auxiliary Systems

The auxiliary systems are based on VMware vSphere 4 Cluster and Cluster RHCS. The auxiliary systems are responsible for the internal management of the FCSCL, mail, monitoring, deployment management, queue management, etc.