Offload features of the original firmware

Hi,

I'm trying to make sure I find all parts of the firmware that Cavium / ITUS had opensourced.
One thing that's giving me headaches going through the original firmware is i'm uncertain if they ever really hit the 'turbo' button on the CPU.

For example, ITUS integrated Snort, but I so far haven't found any libraries linked into the SNORT libraries that seem like they were used to use the CPU HFA offload with Snort. But that would be the exact thing that you would would expect with an Octeon. I also went through the OpenSSH/OpenSSL stuff (a bit) and I don't get the impression that ZLIB or SSL offload is used with OpenSSH.
I don't get the impression either that it's all turned on in OpenSSL - at that point I basically hope that I'm just crazy or blind (hopefully not both) but I try openssl engine, I look for a /dev/cryptodev or such, and I see little. I look at /proc/crypto and I see no telltale signs that some Octeon-specific stuff is called.
Once I got working networking and not just serial I'll run the openssl benchmarks to keep that for later review.

I see in /proc/config.gz that all Octeon related stuff is enabled (phew) and I hope that somewhere the iptables offload stuff is active. I think I saw some of that at least. Don't recall.

Can someone still remember or share some historic insight if generally the shield _did_ use all Octeon offload engines or maybe Cavium tricked them by saying "we support OpenWRT on this CPU" but not completing the other direction "OpenWRT supports everything on this CPU - and it is enabled on the shield?"

Especially with regards to rule offloadf (HFA) i got strong doubts.

It would probably need to be in the first one here.

root@Shield:/# ls /usr/lib/snort_dynamic*/*0.0.0
/usr/lib/snort_dynamicengine/libsf_engine.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_dce2_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_dnp3_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_dns_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_ftptelnet_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_gtp_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_imap_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_modbus_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_pop_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_reputation_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_sdf_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_sip_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_smtp_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_ssh_preproc.so.0.0.0
/usr/lib/snort_dynamicpreprocessor/libsf_ssl_preproc.so.0.0.0



I'm gonna go dig through the sources to find more info, but if anyone already looked at those it would help.
Goal is to not miss anything that was GPLed in some way so I can add it to my CI scripts and then later people can use that to port it forward to all Octeon variants.
For the Octeon9K/10K there's some documentation in marvell's dev docs, based on Intels hypersearch something something. So I think at least inside Palo Alto / Cisco sourcefire etc. those features were always around and used, UBNT I think never managed to figure out how to do (some) of it, thus the failed USG-XG-8 DPI implementation. But other's failures don't need to be our concern, we just need to know what we can find references for and accordingly make usable without the non-existant vendor's support. :-)

If someone has been in contact with the ITUS people during the kickstarter or after that's also interesting, of course. you can message me in private if you think someone is a good contact and won't be annoyed.

The OG firmware was a hack and a half and never took full advantage of the equipment.

Snort is available as a package in mainline Openwrt, I'm working on Suricata, but rust-Lang took forever and still isn't great.

For those who do rust-Lang, mips64-openwrt-linux-musl rs a valid tier 3 tuple target.

I've got the original firmware, and there is a recovery kernel image in /dev/mmcblk1p1 along with the initramfs for all three spots, though mainline openwrt supports only the router selection.

Feel free to join the discord. https://discord.gg/8MNhf6VaGY

Edit: Feel free to hit me up. Software offloading is enabled, no hardware support, and the entire branch is Octeonplus 1.5 tagged for compatibility sake, not Octeon3. I am listed as the device maintainer still, but we are limited to branch changes that can be made

Hi,


I'll join the discord, sure!

Trying to reply...
So first of all - good to know that they never properly used the hardware. That means I'm not blind and that I don't have to preserve their image like it were made of gold. I have no promises at the moment for any snort-side development, but I at least will keep an issue open somewhere that says 'look into octeon HFA and snort via intel hypersearch and octeon10k'. likely I'll wait till I get a Solidrun C9K or a Asterfusion C10K device or a real C10K NIC, build their snort examples there and then figure out what's missing to do the same on old octeons, or where to find the right research-paper-authors with the code.

I tried to figure out how the Itus' shield front switch actually works, I can't find it in the u-boot env. Very confusing still but likely that's just my health, slow brain etc.
I'll try to boot into your kernels tomorrow, it has the switch in the wrong position at the moment and/or a problem with the PoE adapter that's powering it or a bad cable. whatever it is, it doesn't receive anything. Afterwards, I can try booting stuff. This little one is the MVP for trying/booting kernels.
I got some issues to solve which will all be easier with the shield:
- USG-XG-8 PCIe hangs, also on LiquidIO III NICs (probably found the missing patch or that it's incomplete still)
- link speed issue on LiquidIO III NICs (probalby found the bug, just matter of tiredness to continue)
- serial console setup issues (wears my energy)
- ThunderX boards not loading u-boot or booting from SD (stupid)
- LiquidIO III NICs can't replace bootloader (probably found the cause for that, just need testing)
- Setting up fw_printenv in Linux to access the flash (PITA!!!)
- Bisect the Octeon OpenVSwitch image with binwalk, find out what's inside and what they did
(ideally, have OpenVSwitch do some hardware offloading on OpenWRT, too)
- PTP1588 examples were confusing when I tested them



My 'goal' is to free all the LiquidIO III 10/25g cards and document for people how to use them.
I wanna have my own setup with local OS running on the cards, building with -march=octeon2 -mtune=octeon3 on alpine with enabled fpu support. endianness i don't know or care what is better.
mostly I focussed on getting good CI jobs working, cleaning the SDK builds and ideally moving part of the SDK to use openembedded.
In theory, if I do that right, we end up with stabilized toolchain etc and an easer way to enable the hw offload stuff.
I got no embedded dev background, some system integration, so there's a few areas that are blanks but I know the various posts with patches that enabled the crypto features for openwrt on octeon. So I think it should turn out ok...

I want to have fastest possible implementation for LiquidIO III NICs and such hardware, and also OcteonTX. But anything that is sufficiently cleaned up will then also work for octeon1+ again.
I still have things I need to clarify with the 3-tuple for Alpine. It'll not be a supported flavour so it can be named what seems correct.
but I'm not sure yet what that is.
I got a sticky note on my table for it.
cnMIPS64v3 ?
mips64octeon
cnMIPSIII

I know mips64octeon has been used in some projects.
The marketing called the arch cnMIPS64
but that doesn't differentiate well
GCC calls it octeon2/octeon3

So i'm somewhere with

linux-alpine-misp64octeon-muslhf or variants. I'm glad you told me what worked for Rust.

As for distro, I want to base it on Alpine since that's what I use most and can most easily boot to RAM. Downside Alpine wants to break some things in 3.26, maybe then I need to fork away completely. But I think everything I do will be OpenWRT compatible or identical to how it's in OpenWRT.
It's just that my target is mostly things with more cores.


I'll ping you proper in discord when I do something meaningful again, like change the switch position on the shield haha

acutally, what does this mean:

"Software offloading is enabled, no hardware support"

oh and I went through some more datasheets, if one has the tools for reballing, the CPU socket is the same BGA628 among the different models of CN70xx/CN71xx.
One can in theory grab a SonicWall TZ400/TZ400W (800MHz),TZ500 (1GHz) or TZ600 (1.4GHz) or NSA2650 (1.6GHz) and rip out the CPU and put it in the shield as a quadcore upgrade. with some cooling upgrade obviously. but still fun to imagine.
RAM seems to be 8GB max (16GB+ officially). I think if one goes with the 800MHz/1GHz CPUs and same speed/type RAM chances aren't too bad. I'll save that for when I'm retired.



for snort I at least found some howtos

1. for using ARM vector instructions

https://learn.arm.com/learning-paths/servers-and-cloud-computing/vectorscan/install/

2. setting up the intel thing

https://github.com/chenchix/snortinstaller

here as I said I think the best way is to work backwards from intel's implementation and from a C9K based ThunderX2 and from there try to stand up HFA offloading. I think one can also reach out to Sandia for their example DPI code.
What's driving me mad, not just mildly is how little there's directly with regard to Octeon on this. I know it's because they kept it all closed source till they finally paid the price, but still it's really very very very little visible implementations.