The old Cisco Topspin 120 are cheap infiniband switches, but very noisy. You can get one for roughly $200 on eBay. If you're planning for cheap VSAN lab, it might be your interest to read how to make the device more quiet. I'm using one of those in my lab for a backend storage network traffic. While I'm waiting for some more parts I ordered to complete the lab, I was horrified with the noise the device is making. Unless you have a garage or basement which you don't live in, it's ok, but otherwise you must do something.
Update: All wrong. The fans works, but the Topspin cannot control them. The fans needs to have 3 wires. If not the switch just shuts down the OS in the switch because he sees no feedback (RPM) from the fans…. -:(. So I went and installed the beast in another room…
Update 2: I finally got a Noctua fans which do have 3 wires, but needs reorder. It works and it provides similar experience as in this video… -:). See this post for the final setup.
To see the difference I've recorded a short video where you'll see the Before and After… In fact the after, you won't hear almost anything… -:).
I had a little trouble to make it work as the fans I ordered were two pins only while the board had 3 pin connector. Luckily the fans came with an adapter 2 to 3 pins.
The thing is that the pins were not at the correct place and I had to seek with a voltmeter the right voltage coming out of the board and change the pins on the connector supplied with the fan to match the correct place. Nothing really problematic if you know how to use voltmeter….
Here is the quick video when after, you won't hear anything….but the fan is running. I had to completely open the switch and get my hands “dirty”. Also, the fans I ordered were more slim than the original ones and the original attachments did not work so I had to use small 3 mm bolts to attach the fans to the case.
Best to watch in HD, as usually…
I should have all the parts by next week so the full lab shall be up and running.
VSAN in the Homelab – the whole serie:
- My VSAN Journey – Part 1 – The homebrew “node”
- My VSAN journey – Part 2 – How-to delete partitions to prepare disks for VSAN if the disks aren’t clean
- Memory Channel per Bank getting non active when using PERC H310 Contoler card – Fixed!
- Infiniband in the homelab – the missing piece for VMware VSAN
- Cisco Topspin 120 – Homelab Infiniband silence – (This post)
- My VSAN Journey Part 3 – VSAN IO cards – search the VMware HCL
- My VSAN journey – all done!
- How-to Flash Dell Perc H310 with IT Firmware To Change Queue Depth from 25 to 600
AlexMercer says
Could you elaborate some more on the configuration of the switch and the adapters? I’ve got a switch up and running, started a SM on it, so there is no need for a software SM, running on the machines, but I am not sure how to get the nodes to be in the same partition as the switch. How do I configure the HCAs ? I’ve got the OFED stack and mellanox drivers, removed the software subnet manager, since I’m using the one with the switch.
Thanks in advance.