We Developing a cloud infrastructure management system that should support hybrid technology (hypervisor) including For VMs - Kvm/Qemu and for container firecracker/kata-containers, Lxd/Lxc for hosting VDI and our online gaming platform.
It should be highly scalable and support multi region/Zone (federated deployment) for managing thousands of nodes at a time in production
Currently we have Opennebula, hashicorp nomad and Kubernetes/Openshift in mind which supports multiple hypervisors as well as federated deployment.
But still haven’t finalised any of this technology because we are facing multiple challenges in each of these technology.
Having no proper documentation or any dedicated course available
Our Problem -> we want to connect all our containers/VMs across cluster deployed in different Nodes in the nomad cluster. In my current Kubernetes cluster, all pods can communicate to each other based on IP address no matter they are deployed in different machines (worker) using CNI or SDN (Vxlan, Juniper Contrail or Calico / Cilium / Funnel).
The same thing I want to achieve in Nomad Cluster not only for docker containers but other plugins also including firecracker, qemu, etc.
What we want -> All Resources (Container, MicroVm, Vms) should be able to communicate with each other based on IP Addresses under the same subnet across the entire cluster| Simply speaking we want Custom SDN Solution in our nomad cluster.
What I am trying to do actually - I trying to make my own private cloud that can support multiple types of technology including docker, microVM, LXC/LXD container, and KVM as like Opennebula. with the help of the Nomad cluster.
Having no official support kvm/qemu and firecracker / Kata container and not sure about federated deployment.
We had created prototype application using Kubernetes engine which works fine but It only support plain docker containers only not Firecracker and Virtual Machines (Kvm/qemu).
Kubernetes and OpenShift have having support for cluster networking (SDN/CNI) by default (Calico/Cilium)
Due to lack of proper documentation, we have only tested KVM Nodes only.
We want to test firecracker and other container (Lxd/Lxc).
Networking in OpenNebula OpenNebula also having support for cluster networking (SDN) by default (802.1Q VLAN, VXLAN and Open VSwicth) but haven’t tried yet.
Open Nebula can is scalable using federated deployment
Please checkout document where i compare all these technology based on capabilities
*What our Expectation*
we need your team’s expertise (Consultancy) for nominating and developing the right cloud infrastructure that can fulfil our needs. Once we nominated and developed the right technology, we also need your support to deploy the desired solution on production servers with best security and scalability practices.