Skip to content
View c-r-h's full-sized avatar
πŸš΄β€β™‚οΈ
Building an SMB3 Offload Engine: https://gitlab.com/ubiqx/zambezi
πŸš΄β€β™‚οΈ
Building an SMB3 Offload Engine: https://gitlab.com/ubiqx/zambezi
Block or Report

Block or report c-r-h

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
c-r-h/README.md

Header

🍨 Ice Cream Code

If you're looking for code, there's more over on GitLab. I am currently focused on Zambezi, which is an SMB3 Offload Engine for SmartNICs.

I started using GitLab a few years ago. It's not as flashy as GitHub, but I learned a few tricks and got used to it. Moving back and forth between the two is 90% easy and 10% annoying corner cases that trip me up.

πŸ› οΈ SMB/CIFS, SMB2, and SMB3

Samba Team Member
I write code and share knowledge about Windows core network protocols, particularly Server Message Block (SMB).

  • Implementing CIFS
    A few years back, I wrote a book about CIFS (aka SMB1). Since then, Microsoft has deprecated SMB1, but the book is still relevant and I, for one, still use it as a reference.
  • [MS-CIFS] and [MS-SMB]
    Fewer years back, based on the strength of my CIFS book, I was asked to put together a team to write the SMB1 specifications for Microsoft. We wrote [MS-CIFS] from scratch and overhauled [MS-SMB]. The latter volume represents the additions and changes made to SMB1 starting with Windows 2000. Once again, SMB1 has been rightly deprecated by Microsoft, but it's still in use in a lot of environments so it's really important to have a solid reference.
  • SMB3 Offload Engine for SmartNICs and DPUs
    These days I'm working on SMB2/3 integration with distributed file systems, and on an SMB3 Offload Engine. I think it would be cool if low-level encryption, compression, and basic message handling could be moved to a SmartNIC or similar device.
    β€£ Compression and Encryption
    β€£ Multichannel and Multi-protocol
    β€£ Packet marshalling/unmarshalling
    β€£ Proxy and Software-Defined Servers
  • I started fiddling with PeerDist a while back, but have not spent time on it in...well...years. I plan to get back to it. Eventually.

🎁 Trinkets

  • ubiqx Modules
    A small collection of fairly common data structures that you may have implemented in an undergraduate CS class. I wrote these long ago. I reuse them rather than rewriting them over and over again.

🏰 My Fiefdom

  • The Home Network
    The network was once split between business and home. There were two Soekris routers (remember those?) running OpenBSD's Packet Filter (pf(4)), with routing in between. It was a great setup, and I ran things that way for years. More recently, I've combined down to a single network. I run OpnSense on a Protectli Vault. OpnSense is easier to manage day-to day, but I miss being able to fine tune my firewall to meet my own picky specifications and preferences.
  • Firewall Futures
    • I used to have uplinks to two disreputable behemoth ISP services. That's down to one, now. Two would permit load balancing and failover, which I would like to try.
    • I want to collect data at the firewalls, and coordinate with (anonymized) data from other firewalls to look for patterns. You know, Big Data/AI stuff.
  • Reverse Proxy Webserver
    Slowly, but committedly, I'm spending free cycles building up a home web server with the goal of running a reverse caching proxy elsewhere on the 'net and connecting the two via VPN. I'm sharing this effort with a family member, who is an accomplished Software Developer in her own right. Proud to be part of a cool family.
  • File Servers
    I have a set of Linux MD RAID based Samba fileservers that I run in my basement. They're built on CentOS7 Minimal, and I add what packages I need if I need them but these systems don't need much. I use these systems for business and personal data storage, and sometimes a bit of code testing. I will be switching to Ubuntu Server as CentOS7 approaches end of life.

πŸ’‘ Ideas

  • SMB3 NAS Offload Engine
    Top Project: Zambezi
    The goal is to develop and standardize a sensible interface between the SMB3 Syntactic and Semantic layers, making it possible to offload the Syntactic part of the stack to a SmartNIC or DPU. The interface layer may also have relevance in direct memory-to-memory data transfer systems such as SDXI.
  • SMB3 Python Toolkit
    Carnaval is an SMB2/3 toolkit in Python, aimed at building test cases and simple tools quickly and easily. It also provides support for NBT transport. Carnaval is written in Python 2, not Py3. The project has languished a bit over the years, but as Zambezi progresses the need for such a toolkit only increases.
  • Secure Overlay Network
    Back when I designed networks for the University of Minnesota, I became enamored of Resilient Overlay Networks. If you mix in some VPN capabilities, erasure codes, and clever routing algorithms, you might have something really cool--particularly in these work-from-home times. There are certainly extant examples, like tinc, which supports mesh networking. I'm looking into tinc, and others, to see what fits best. OpnSense supports tinc, so perhaps the job is done, but I feel the need to dig into the code a bit.
  • Wide Area Distributed Key/Value store Key/Value stores are generally built for performance. I'm interested in what happens when that model is extended over a wide area network, particularly a resilient mesh, as described above. I think we'd need to add in some redundancy and possibly some form of journaling so that fenced nodes can recover cleanly.
  • PeerDist This one keeps coming up time and again, and it's entangled with so much else...
    PeerDist is the protocol suite at the core of BranchCache, which is a distributed caching system that Microsoft uses for web content, SMB3 files, and for software updates (see BITS). I have created test code for working with PeerDist and BITS. The next step would be polish up the PeerDist code, add PeerDistv2 support, and write the code for an Open Source Hosted Cache. PeerDist combines all sorts of cool stuff, like Protocols, Data Storage, Key/Value Lookup, and Distributed Computing.

πŸ”© Cool Hardware

My desktop system is twelve years old. My laptop is ten years old. My Chromebook is only six months old, but that's because the screen shattered on my first Chromebook so I bought a new one. I'm not really into hardware. I'm into making software work well even on substandard hardware.

That said, there are some cool products that I've been eyeing:

  • The Kobol Helios64 would be a great little platform for testing out some of my SMB3 Offload ideas. It'd also work as yet another home storage system.
  • I like the look of the Seeed Odyssey SBC. I'm not sure yet exactly how I would use it, and how it would be an improvement over the Raspberry Pi I already have.
  • So... at the core of many a SmartNIC is a DPU; Data Processing Unit. The DPU is quickly becoming a focus of attention in the industry, and I expect to see them invade motherboard territory in the not-too-distant future. This is the perfect environment for an SMB3 Offload engine (and NFS offload as well).
    • Fungible has a stand-alone DPU which, if I read their marketing material correctly, would be added to a computer mainboard for a closer connection with the CPU, memory, etc.
    • More recently, NVIDA announced their own DPU aimed at networking, storage, and security.

🧠 Cool Software

There's the normal stuff I do, including shell scripting and C coding and such-like, and then there are the bright shiny things that catch my attention. I just wish I had time to play with them.

  • Ultibo
    ...is a modified Free Pascal compiler that targets the Raspberry Pi. It includes an extensive set of libraries, and the output does not require an underlying OS. Cool, weird, fun, and intriguing.
    Years ago, Object Oriented Pascal was my favorite programming language, and I was very happy when I stumbled across the Free Pascal compiler. Unfortunately, I have had neither the time nor the driving reason to get back into it. Perhaps this is the excuse I need.
  • Crystal
    I like object oriented programming languages, but I've been disappointed by some of the more popular ones out there. I'd like to give this one a test drive.

πŸ‹ A Challenge

I have been kicked hard in the bike shorts by proprietary cycling computers. The ones I have tried are closed-source and have no published API. When things go wrong, there is no recourse. This is the Achilles' heel of these expensive devices. All of the problems I've had with them, including data loss and other frustrations, are due to their closed, proprietary nature.

So I am looking for an open option. Here's what I've found so far:

  • Open Cycling Computer
    The project was quite far along, but appears to have fallen stagnant. Dang shame, given how much work went into it. Perhaps it can be revived and updated, given a little time and attention.
  • Open Bicycle Computer
    Another project that has seemingly dried up. Worth reviewing, though.
  • Open Source Bike GPS
    A much more active Open Bike Computer project. There are links in the ReadMe to hardware information. This one looks quite promising.
  • Pyloton
    ...is much less ambitious than the other projects on the list, but I could learn a thing or two from the parts and pieces list alone.
  • Karoo-API
    The folks at Hammerhead, who put out the Karoo bike computer, have started creating toolkits. It appears to be very new code, much of it forked from other projects (which is fine). I need to watch and consider getting involved.
  • Golden Cheetah
    Ride file analytics. This would pair nicely with an open GPS device.

πŸ“– Further Study

There isn't much time, given everything listed above, but it would be fun to do a bit more learning.

  • Android Development
    You know... apps and such.
    A long time ago I co-founded a project called jCIFS to create a Java client for SMB1. It wound up in a lot of places, from supercomputers to home media players, but was particularly useful as an SMB1 engine for Android Apps. Sadly, we all got distracted by other projects and never added SMB2/3 support to jCIFS (but see: jcifs-ng).
  • A bit more about Docker
    I've done more than dabble, but I am really just an end-user. The Windows Protocol Test Suites (WPTS) can be run from inside a Docker Linux image, so I guess it's a good time to get practical experience.
  • Kubernetes, Ansible, Terraform... yeah, I need to fly in the cloud some more.

🀯 More Yet

β˜• I enjoy a decent cup o'tea.
🀺 For much of my life I was an active fencer. (Foil:C, Γ‰pΓ©e:D)
🚴 These days I prefer to ride a bicycle.
You can find me on LinkedIn.
You can find more stuff on GitLab.




$Id: README.md; 2021-06-06 16:28:36 -0500; crh$

Pinned

  1. ubiqx-org/Carnaval ubiqx-org/Carnaval Public

    A Python implementation of the SMB protocol suite.

    Python 11 4

  2. ubiqx-org/Modules ubiqx-org/Modules Public

    Simple tools you should never need to write yourself.

    C