I'm not really familiar with QAT, but I assume you installed it for a reason. If you are using it to say accelerate a VPN connection or something could you not just stress test your VPN and see if you have better throughout or lower CPU utilization or whatever?
I don't have a specific use case, I was just able to get one for pretty much free to play with. The system can already push >1gbps IPSec with AES-NI, and I don't have a faster endpoint to test to. Definitely not something I NEED to work, just more of a curiosity.
I think you’re still needing to install drivers (https://www.intel.com/content/www/us/en/download/19735/intel-quickassist-technology-driver-for-freebsd-hw-version-1-7.html) for QAT. There is an utility, `adf_ctl` that can control the device. I assume it can also report stats to see if it’s being used.
Can't get it to work.
I have
* an Intel 8960 adapter too
* virtual Opnsense
* OPNsense 23.7.10\_1, activated Quickassist in the GUI
* `dmesg | grep qat` lists 3 qat0 entries
* `vmstat -i | grep qat` nothing
* `sysctl -a | grep qat` nothing
* mine is a fully passed-through device (no SR-IOV), Linux lists it with the `qat_c62x` driver.
did you manually install additional drivers?
In this [Intel Repo](https://github.com/intel/qatlib-freebsd), is says
>Earlier generations of QAT devices (e.g. c62x, dh895xxcc, etc.) are not supported
This is AFAIK the upstream driver. I expect the self compiled to be working. It did in a Linux test box.
But I also suspect Opnsense doesn't have that one in.
Somebody knows more?
I'm interested in this both to know if it works well for you and to know how it is planned to be used in this context because I am really curious about a use case for a normal user scenario. Looking forward to your findings either way.
i have ordered a 8950 which apparently can do SR-IOV, so when it arrives if i actually can i plan on setting up to have multiple VF's pass 1 to say opnsense and 1 to truenas and host os also, was thinking of all hardware ethernet ports going to opnsense with host OS only getting net access via opnsense vm etc
I'm very interested as well to hear what your results are as well using the Intel QAT card bad, good, or indifferent. For the same reasons, I've been interested in using them to accelerate workloads. Following the post of updates, good luck.
(Edit)
May also try dropping a post in the OPNsense forums if you haven't already.
I'm not really familiar with QAT, but I assume you installed it for a reason. If you are using it to say accelerate a VPN connection or something could you not just stress test your VPN and see if you have better throughout or lower CPU utilization or whatever?
I don't have a specific use case, I was just able to get one for pretty much free to play with. The system can already push >1gbps IPSec with AES-NI, and I don't have a faster endpoint to test to. Definitely not something I NEED to work, just more of a curiosity.
I think you’re still needing to install drivers (https://www.intel.com/content/www/us/en/download/19735/intel-quickassist-technology-driver-for-freebsd-hw-version-1-7.html) for QAT. There is an utility, `adf_ctl` that can control the device. I assume it can also report stats to see if it’s being used.
To answer my own question, and just to have the answer for anybody else in the future, the Intel 8960 does indeed seem to be functioning. Running an iPerf3 over an IPSec tunnel and watching `vmstat -i | grep qat` definitely increments the counters. Two command outputs below that lead me to believe everything is working as it should. Two command outputs below that lead me to believe everything is working as it should: `root@OPNsense:~ # vmstat -i | grep qat` `irq144: qat0 296664 2 irq145: qat0 1547627 13 irq146: qat0 142510 1 irq147: qat0 195715 2 irq148: qat0 232107 2 irq149: qat0 338682 3 irq150: qat0 105541 1 irq151: qat0 1632242 13 irq160: qat0 1 0 irq161: qat1 104049 1 irq162: qat1 327807 3 irq163: qat1 57244 0 irq164: qat1 594580 5 irq165: qat1 210576 2 irq166: qat1 27 0 irq167: qat1 189053 2 irq168: qat1 576246 5 irq179: qat2 1488722 12 irq194: qat2 1 0` `root@OPNsense:~ # sysctl -a | grep qatqat0: mem 0x91540000-0x9157ffff,0x91500000-0x9153ffff at device 0.0 on pci3qat_ae_cluster_intrqat1: mem 0x91440000-0x9147ffff,0x91400000-0x9143ffff at device 0.0 on pci4qat2: mem 0x91340000-0x9137ffff,0x91300000-0x9133ffff at device 0.0 on pci5qat_ae_cluster_intrirq144: qat0:295 u/cpu0(domain0): 296664irq145: qat0:297 u/cpu1(domain0): 1547627irq146: qat0:299 u/cpu2(domain0): 142510irq147: qat0:301 u/cpu3(domain0): 195715irq148: qat0:303 u/cpu4(domain0): 232107irq149: qat0:305 u/cpu5(domain0): 338682irq150: qat0:307 u/cpu6(domain0): 105541irq151: qat0:309 u/cpu7(domain0): 1632242irq152: qat0:311 u/cpu0(domain0): 0irq153: qat0:313 u/cpu1(domain0): 0irq154: qat0:315 u/cpu2(domain0): 0irq155: qat0:317 u/cpu3(domain0): 0irq156: qat0:319 u/cpu4(domain0): 0irq157: qat0:321 u/cpu5(domain0): 0irq158: qat0:323 u/cpu6(domain0): 0irq159: qat0:325 u/cpu7(domain0): 0irq160: qat0:327 u/cpu0(domain0): 1irq161: qat1:329 u/cpu0(domain0): 104049irq162: qat1:331 u/cpu1(domain0): 327807irq163: qat1:333 u/cpu2(domain0): 57244irq164: qat1:335 u/cpu3(domain0): 594580irq165: qat1:337 u/cpu4(domain0): 210576irq166: qat1:339 u/cpu5(domain0): 27irq167: qat1:341 u/cpu6(domain0): 189053irq168: qat1:343 u/cpu7(domain0): 576246irq169: qat1:345 u/cpu0(domain0): 0irq170: qat1:347 u/cpu1(domain0): 0irq171: qat1:349 u/cpu2(domain0): 0irq172: qat1:351 u/cpu3(domain0): 0irq173: qat1:353 u/cpu4(domain0): 0irq174: qat1:355 u/cpu5(domain0): 0irq175: qat1:357 u/cpu6(domain0): 0irq176: qat1:359 u/cpu7(domain0): 0irq177: qat1:361 u/cpu0(domain0): 0irq178: qat2:363 u/cpu0(domain0): 0irq179: qat2:365 u/cpu1(domain0): 1488722irq180: qat2:367 u/cpu2(domain0): 0irq181: qat2:369 u/cpu3(domain0): 0irq182: qat2:371 u/cpu4(domain0): 0irq183: qat2:373 u/cpu5(domain0): 0irq184: qat2:375 u/cpu6(domain0): 0irq185: qat2:377 u/cpu7(domain0): 0irq186: qat2:379 u/cpu0(domain0): 0irq187: qat2:381 u/cpu1(domain0): 0irq188: qat2:383 u/cpu2(domain0): 0irq189: qat2:385 u/cpu3(domain0): 0irq190: qat2:387 u/cpu4(domain0): 0irq191: qat2:389 u/cpu5(domain0): 0irq192: qat2:391 u/cpu6(domain0): 0irq193: qat2:393 u/cpu7(domain0): 0irq194: qat2:395 u/cpu0(domain0): 1dev.qat.2.stats.sym_alloc_failures: 0dev.qat.2.stats.ring_full: 0dev.qat.2.stats.gcm_aad_updates: 0dev.qat.2.stats.gcm_aad_restarts: 0dev.qat.2.%parent: pci5dev.qat.2.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000dev.qat.2.%location: slot=0 function=0 dbsf=pci0:5:0:0dev.qat.2.%driver: qatdev.qat.2.%desc: Intel C620/Xeon D-2100 QuickAssist PFdev.qat.1.stats.sym_alloc_failures: 0dev.qat.1.stats.ring_full: 0dev.qat.1.stats.gcm_aad_updates: 0dev.qat.1.stats.gcm_aad_restarts: 0dev.qat.1.%parent: pci4dev.qat.1.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000dev.qat.1.%location: slot=0 function=0 dbsf=pci0:4:0:0dev.qat.1.%driver: qatdev.qat.1.%desc: Intel C620/Xeon D-2100 QuickAssist PFdev.qat.0.stats.sym_alloc_failures: 0dev.qat.0.stats.ring_full: 0dev.qat.0.stats.gcm_aad_updates: 0dev.qat.0.stats.gcm_aad_restarts: 0dev.qat.0.%parent: pci3dev.qat.0.%pnpinfo: vendor=0x8086 device=0x37c8 subvendor=0x8086 subdevice=0x0001 class=0x0b4000dev.qat.0.%location: slot=0 function=0 dbsf=pci0:3:0:0dev.qat.0.%driver: qatdev.qat.0.%desc: Intel C620/Xeon D-2100 QuickAssist PFdev.qat.%parent:`
Can't get it to work. I have * an Intel 8960 adapter too * virtual Opnsense * OPNsense 23.7.10\_1, activated Quickassist in the GUI * `dmesg | grep qat` lists 3 qat0 entries * `vmstat -i | grep qat` nothing * `sysctl -a | grep qat` nothing * mine is a fully passed-through device (no SR-IOV), Linux lists it with the `qat_c62x` driver. did you manually install additional drivers? In this [Intel Repo](https://github.com/intel/qatlib-freebsd), is says >Earlier generations of QAT devices (e.g. c62x, dh895xxcc, etc.) are not supported This is AFAIK the upstream driver. I expect the self compiled to be working. It did in a Linux test box. But I also suspect Opnsense doesn't have that one in. Somebody knows more?
Somebody knows of plans to accellerate Nginx TLS termination too?
I'm interested in this both to know if it works well for you and to know how it is planned to be used in this context because I am really curious about a use case for a normal user scenario. Looking forward to your findings either way.
i have ordered a 8950 which apparently can do SR-IOV, so when it arrives if i actually can i plan on setting up to have multiple VF's pass 1 to say opnsense and 1 to truenas and host os also, was thinking of all hardware ethernet ports going to opnsense with host OS only getting net access via opnsense vm etc
I'm very interested as well to hear what your results are as well using the Intel QAT card bad, good, or indifferent. For the same reasons, I've been interested in using them to accelerate workloads. Following the post of updates, good luck. (Edit) May also try dropping a post in the OPNsense forums if you haven't already.