ovirt and gluster_hyperconvergence_devconf-2016
TRANSCRIPT
Devconf.cz, February 2016
oVirt and Gluster Hyperconvergence
Ramesh NachimuthuSoftware EngineerRed HatDevconf.cz, February 2016
Devconf.cz, February 2016
Agenda
● Architecture of oVirt● Hosted Engine● Gluster in a nutshell● Hyper converged infrastructure
● How it looks● Setup● Management
● Recent enhancements● Conclusion
Devconf.cz, February 2016
Ovirt Architecture
oVirt is a virtualization platformto manage virtual machines,storage and networks
● Engine (ovirt-engine)Manages the oVirt hosts, and allows system administrators to create and deploy new Vms
● Host Agent (VDSM)oVirt engine communicates with VSDM to manage the Vms, storages and networks
Clu
ster
s
Storage ServerSto
rage
Eng
ine
Storage Server
Cluster Hosts
Devconf.cz, February 2016
Ovirt Storage Architecture
● Storage Domain● Centralized storage system
(images, templates, etc.)
● A standalone storage entity
● Stores the images and associated
Metadata
● Only real persistent storage for
VDSM
● Used for synchronization (sanlock)
● Storage Types● NFS, FCP, iSCSI
● Glusterfs
Clu
ster
s
Storage DomainSto
rage
Eng
ine
Storage Domain
VDSM Hosts
Devconf.cz, February 2016
Hosted Engine
● Ovirt Engine running inside a
VM
● Highly Available
● Can be migrated to a
different host
● Special Agent for Monitoring
● Bootstrap deployment
neededC
lust
ers
Storage Domain
Sto
rage
Poo
lE
ngin
e
Storage Domain
VDSM Hosts
Devconf.cz, February 2016
GlusterFS Architecture
● GlusterFS is a general purpose scale-out distributed file-system
supporting thousands of clients
● Aggregates storage exports over network interconnect to provide
a single unified namespace
● File-system completely in
userspace, runs on
commodity hardware
● Layered on disk file
systems that support
extended attributes
Devconf.cz, February 2016
GlusterFS Bricks
● A brick is an export directory located on a specific node
(e.g. host-01:/srv/fs1/brick1)
● Each brick inherits limits of the underlying file-system
● No limit on the number bricks per node (as best-practice each
brick in a cluster should be of the same size)
/srv/fs1/brick1 /srv/fs1/brick2/srv/fs1/brick2
File-System: /srv/fs1 /srv/fs2
Block Device: /dev/dm-1 /dev/dm-2
Devconf.cz, February 2016
GlusterFS Volumes
● A volume is a logical collection of bricks
● The mountable entity
● Different types of Volumes
● Distribute, Stripe, Replicate (+ combinations), Quorum
● Type of a volume is specified at the time of volume creation
and determines how and where data is placed
● Accessible via FUSE, NFS, SMB, libgfapi
host01:/srv/fs1/brick1
Volume1
host02:/srv/fs1/brick1 host03:/srv/fs1/brick1
Distribute/Stripe/Replicate
Devconf.cz, February 2016
Fuse Access
QEMU
Kernel VFS /dev/fuse
Kernel
Userspace
GlusterFS
Net
File-System
Gluster Brick
NetKernel
Userspace
Client Server
FUSE Access
● More context switches
Devconf.cz, February 2016
QEMU libgfapi Support
● GlusterFS exposes APIs for accessing Gluster volumes
● Reduces context switches
QEMU
Kernel
Userspace
Net
File-System
Gluster Brick
NetKernel
Userspace
Client Server
libgfapi Access
But see: https://bugzilla.redhat.com/show_bug.cgi?id=1247933
Devconf.cz, February 2016
Hyperconverged oVirt – GlusterFS
● The Data Center nodes are used both for virtualization and serving replicated images from the GlusterFS Bricks
● Engine runs inside a VM (Hosted Engine)
● The boxes can be standardized (hardware and deployment) for easy addition and replacement
● Support for both scaling up, adding more disks, and scaling out, adding more hosts
VM
s an
d S
tora
ge w
ith E
ngin
e in
side
a V
M
GlusterFS Volume
Bricks Bricks Bricks
Experimental?
Devconf.cz, February 2016
Improvements needed for HC
● GlusterFS support re-added to hosted-engine setup● With libgfapi and replica 3 volume
● Ovirt-Engine Appliance● Preconfigured VM with oVirt Engine● Cloud-init based customization
● Shared configuration● All nodes see the same configuration data● Upgrade path from oVirt 3.5
● Management GUI for the oVirt-engine VM and HE
Devconf.cz, February 2016
HC ingredients
● at least 3 virtualization capable hosts (CentOS 7.1+)● 10 GB of temporary space on the primary host● two separate partitions for data (20GB+) on all hosts● DHCP configured to map a MAC address to a fixed IP● DNS configured with A and PTR names for the IP● oVirt release package installed on all hosts
http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm
● Physical console on the primary host or network access and screen package installed
Devconf.cz, February 2016
HC Setup - Gluster Volume
● 3 Node Gluster cluster● 2 Volumes with 3 way replication
● One for Hosted Engine VM and other for regular VMs● Enable Cluster Quorum● Sharding for better self-heal and geo-rep performace● Few more tunings
● network.ping-timeout = 10● cluster.data-self-heal-algorithm = full● performance.low-prio-threads = 32● Group = virt● storage.owner-uid, storage.owner-uid = 36
Devconf.cz, February 2016
HC Setup – Hosted Engine
Packages Required● ovirt-engine-appliance● ovirt-hosted-engine-setup● Vdsm-gluster
hosted-engine –-deploy● 'glusterfs' storage type● OVA image for hosted-engine VM● Clound-init to customize the VM
● FQDN and Mac for hosted-engine VM
More details at http://blogs-ramesh.blogspot.in/2016/01/ovirt-and-gluster-hyperconvergence.html
Devconf.cz, February 2016
HC Setup – Final Touch
● Hosted Engine running with single node Cluster● Enable Gluster service● Add remaining nodes● Create and add the main storage domain
Devconf.cz, February 2016
Enabling Gluster
● Gluster Service support is located in the Cluster properties
● Deploy Hosts with GlusterFS Server support
● Enable Bricks and Volume Management from oVirt WebAdmin and REST-API
● Engine is not taking in consideration GlusterFS on Virtualization Power-Saving policies and Fencing yet
Devconf.cz, February 2016
Adding addtional Nodes
● Simple checkbox during in the Add host dialog
● Host deploy script does everything else auto-magically
Removed at the last moment because of unresolved issues. It can be done through “hosted-engine –deploy” script
Devconf.cz, February 2016
Adding Gluster storage
● Gluster volume management using WebAdmin Portal and REST-API
● Volume Pofiling and Capacity monitoring
● Geo Replication
● Volume Snapshot
Devconf.cz, February 2016
Add Storage Domain
● Master Storage Domain using gluster volume
● Imports Hosted Engine Storage Domain and Hosted Engine VM autmotically.
Devconf.cz, February 2016
Dashboard
Devconf.cz, February 2016
Hosted engine VM management
● Support for editing the Hosted Engine VM
● Memory and CPU allocation, network configuration
● Work in progress..
● Distributed to all hosted engine nodes using OVF file on the storage domain
● Hosted engine daemons pick up the configuration when the management VM is restarted
Devconf.cz, February 2016
Hosted engine management
● Reporting configuration
– State transitions
– SMTP details● Timeout configuration
– Allowed downtime before forced recovery● Host scoring constants and rules
Devconf.cz, February 2016
Recent Enahacements
● Glusterfs Sharding
● Arbiter Volumes
● Replace Brick
● Backup vol file support in libvirt and QEMU-KVM
https://devconfcz2016.sched.org/event/5m1p/improvements-in-gluster-for-virtualization-usecase
Devconf.cz, February 2016
THANK YOU!
http://[email protected]@ovirt.org#ovirt irc.oftc.net
http://www.ovirt.org/Features/GlusterFS-Hyperconvergencehttp://blogs-ramesh.blogspot.in/2016/01/ovirt-and-gluster-hyperconvergence.html