1. 4Oracle Solaris ZonesOracle Solaris KazuyukiSatoSeptember06,2014Corydorasarmatus(Gnther,1868)

2. Agenda @satokaz Oracle Solaris Zones Oracle Solaris Resource Manager Oracle Solaris 11.2 2 3. @satokaz Solaris Solaris Solaris () AQUA LIFE OpenStack Solaris 3 4. SPARC/Solaris Hypervisor HV (HV)Hypervisor (HV)PDomMultiple OS Single OSAppOSServerHypervisor (HV)Hard Partition(PDoms : Physical Domains)Oracle VM Server(LDoms : Logical Domains) Solaris ZonesResource ManagerM6-32 SeriesT-SeriesM6-32 SeriesT-SeriesM6-32 SeriesT-SeriesM6-32 SeriesHVPDomHVPDom 5. Project Blackbox 1 5000 Sun Modular Datacenter5 6. Oracle Solaris Zones Na$veZones6 7. Oracle Solaris Zones () 1960 1 API ABI Solaris API/ABI 7 8. Oracle Solaris Zones Solaris Zones (2003 ) (Security) (Isolation) (Virtualization) (Granularity) (Transparency) 8 9. Oracle Solaris Zones chrootjailschroot Trusted Operating SystemOS Hardware LogicalPartitioning2003 logical partitions for SPARC Ldoms (Oracle VM for SPARC)9 Solaris jails OS 10. Oracle Solaris Zones(Solaris ) 2 (global zone) (non-global zone) 8192 () zone01 zone02 zone03net0:1zcons/usrnet0:2/data/usrnet1/data/usrzoneadmd zoneadmd zoneadmd (zonecfg, zoneadm, zlogin, etc)C C C CVirtualPlatform/datazconszcons 11. Oracle Solaris Zones() users, root, IP addresses, ports, filesystem, ... Possibility for different name services (LDAP, ...) ps, /proc, prstat, ... only show local processes No sharing memory other than local to the zone Private piece of disk for identityapplication Shared read-only to underlying OS (/lib,/usr,...)11 12. Oracle Solaris Zones(sparse root zones)(global zones)(whole root zones)//usr/lib /platform/zones /export/sbin/zone01/root /dev/etc/var/zone02/dev /root/(local,iscsi,fc)/usr/platform/sbin/lib/export/etc/var/opt/opt//usr/platform/sbin/lib/export/etc/var/opt Solaris 10 Solaris 11 13. Oracle Solaris Zones IP (shared IP) Oracle Solaris 10 10/08 IP (exclusive IP) Oracle Solaris 10 8/07 GLDv3NIC Solaris 11 NIC IPIPnet1net0:1net0:0IP IP 14. Oracle Solaris Zones OS (zone02) (zone03)(zone01)(global)PP PPP PPP P 15. Oracle Solaris Zones (privileges) Oracle Solaris 10 11/06 dtrace_kernel proc_zone sys_config sys_devices sys_linkdir sys_net_config sys_res_config sys_suser_compat(zone03)PPP(zone02)PPP(global)(zone01)PPPPPP 16. Oracle Solaris ZonesOracle Solaris 11 ZonesOracle Solaris Zones Solaris IPS NFS IP crossbow Read only Zone Oracle Solaris 10 Zones 17. Oracle Solaris ZonesSolaris 11 Solaris 11 solaris (SPARC/x86) Solaris 11 solaris10 (SPARC/x86,) Solaris10 Solaris 10 P2V Solaris 10 Solaris 10s10zs10zSolaris 11 18. Oracle Solaris Zones /etc/system IPQoS/IPMP/ NFS NFS(zonepath) 19. Oracle Solaris ZonesSolaris Zones Docker Solaris ZonesDockerNamespace Linux NamespacesSolaris Resource ManagerLinux cgroupsZFS, ZFS snapshotAUFSDevice Mapper ThinProvisioningSolaris Linux iptables19 20. Oracle Solaris ZonesSolaris Zones 20zonecfgzones zoneadmzones (install, uninstall, boot, halt,shutdown, attach/detach, clone, etc) zloginzones zonestatzone No Zonedelete createincompletehalt readyreadyinstalleduninstallmarkincompleteuninstall installready boot haltbootmarkincompleterebootconfiguredloginrunningZonezlogin zoneadm zonecfg 21. Oracle Solaris Zones Solaris 11 Zones NIC# zonecfg -z testzone01testzone01: No such zone configuredUse 'create' to begin configuring a new zone.zonecfg:testzone01 createcreate: Using system default template 'SYSdefault'zonecfg:testzone01 set zonepath=/rpool/zones/testzone01zonecfg:testzone01 set autoboot=truezonecfg:testzone01 commitzonecfg:testzone01 exittestzone01Solaris 11 NICnet0global zoneSolaris 11 OS # zoneadm -z testzone01 install# zoneadm -z testzone01 boot ; zlogin -C testzone01[Connected to zone 'testzone01' console][NOTICE: Zone booting up]SunOS Release 5.11 Version 11.0 64-bitCopyright (c) 1983, 2011, Oracle and/or its affiliates. All rights OS 22. Oracle Solaris Resource Manager22 23. Oracle Solaris Resource Manager Oralce SolarisOracle SolarisPPPPPPCPU CPUpset_projpool_projCPU CPUpset_zonepool_zonePPPPPPSolaris 9Oracle Solaris 10+ 24. Oracle Solaris Resource Manager zone.cpu-shares zone.cpu-cap zone.max-swap zone.max-locked-memory zone.max-lofi zone.max-lwps zone.max-shm-memory zone.max-shm-ids zone.max-sem-ids zone.max-msg-ids zone.max-processes 25. Oracle Solaris Resource ManagerCPU 3CPU CPU capped-cpu Oracle Solaris 10 5/08 CPU dedicated-cpu Oracle Solaris 10 8/07 CPU /capped-cpu 26. Oracle Solaris Resource Manager CPU *1 FSS (Fair Share Scheduler) CPUCPU TS (Time Sharing) Oracle Solaris CPUCPU CPU CPU CPUCPU CPU CPU CPUpset_zonepool_zone (FSS)pset_defaultpool_default(zone02)(zone01)(global)*1 : OS 27. Oracle Solaris Resource Manager zone010% 100%zone02zone01(global)CPU CPUpset_defaultpool_default20% 80%100% 0%(zone01)(zone02)cpu-shares : 20 cpu-shares : 80CPU CPUpset_zonepool_zone (FSS) 28. Oracle Solaris Resource Managercapped-cpu 1 CPU100%CPU() zone0150%CPU CPUpset_defaultpool_default (FSS)(global)1 / 2 (50%)(zone02)(zone01)ncpus : 1 29. Oracle Solaris Resource Managerdedicated-cpu CPU() /capped-cpuzone02(zone02)CPU CPUSUNWtmp_zone02SUNWtmp_zone02CPU CPUpset_defaultpool_default(global)(zone01)100%2 CPUncpus : 2 30. Oracle Solaris Resource Manager CPU /dedicated-cpu25% 37.5%50% 25% 25%(global)CPU CPUpset_defaultpool_default37.5%(zone02)CPU CPUCPUpset_zone(zone01)pool_zone (FSS)CPUcpu-shares : 50 cpu-shares : 50 31. Oracle Solaris Resource Manager capped-memory Oracle Solaris 10 8/07 (physical) (swap) (locked) 1GB1GB 5GB 2GB(global)(prod)(devel)(global)1GB 2GB 3GB(prod)(devel)physical : 2GBPPPP PPP PPPPPP PPP PP3GBswp mem swp mem 32. Oracle Solaris Resource ManagerSysmte V IPC /etc/system max-shm-memory (shmsys:shminfo_shmmax) System V max-shm-ids (shmsys:shminfo_shmmni) System VID max-sem-ids (semsys:seminfo_semmni) ID max-msg-ids (msgsys:msginfo_msgmni) ID 33. Oracle Solaris Zones Oracle Solaris Zones Oracle Solaris Zones + Oracle Solaris Resource Manager 33 34. Oracle Solaris 11.2 KernelZones34 35. Native ZonesOracle Solaris 11.2 Live Zone Reconfiguration (LZR) CMT aware Zones and Pools CPU// OpenStack 35 36. Engineered for CloudOracle Solaris 11.22014731 2014/7/31 Oracle Solaris Cluster 4.2 2014/8/5 37. Solaris Kernel Zonesn37 Kernel Zones Type 2 Solaris Zones (HW ) Solaris Zones 38. Solaris Zones SuspendResume Live Migration 38Oracle Solaris 11.2 39. Solaris Kerenl ZonesOracle Solaris 11.2 For SPARC SPARC T4: System Firmware 8.5.0 SPARC T5, M5, M6: System Firmware 9.2.0 For x86 BIOS/EFI CPU 8GB ZFS ARC(Adaptive Replaecement Cache) brand/brand-solaris-kz virtinfo # virtinfo (SPARC T5-8 )NAME CLASSlogical-domain currentnon-global-zone supportedkernel-zone supported Kernel Zones CPU1 (virtual-cpu)Memory2GB ()Storage16GBNetworkexclusive , Single NIC, Random MAC 40. 40 41. Solaris Kernel Zones Best-of-BreedMigra$on // () (CIFS ) Solaris N on Solaris N+ Solaris 11.2 Solaris 11.3 Solaris 12 8 Oracle Confidential: Need to knowZone Zone u1 Zone u2 New generation of Zones Own version of kernel Allows patching/upgrading of zonesto different OS releases individually Antfarm zones can be suspended/resumed Live Migration possible! Admin/Config/ResourceManagement interfaces exactlysimilar to zones One set of resource controls andpolicies for allDifferent update levelsZone Zone ASystem A System BLive Migration 42. Solaris Kernel Zones zoneadm suspend kernel zones suspend resource property zonecfg z KZ-Zones info suspend supend AES-128-CCM /dev/random suspend suspend zoneadm boot suspend Zone 42 43. kzhost zvmm kzhost kernel zone CPU I/O Type 2 zvmm (zone virtual machine monitor) CPU ioctl 44. zvblk disk I/O zvnet zvterm , zlogins zvsdir () zvcntrl apic, bus, timers,management, etc OSkernel zones (Solaris 10 )root@solaris-s12:~# prtconf -DSystem Configuration: Oracle Corporation i86pcMemory size: 2048 MegabytesSystem Peripherals (Software Nodes):i86pc (driver name: rootnex)scsi_vhci, instance #0 (driver name: scsi_vhci)zvnex, instance #0 (driver name: zvnex)zvcntrl, instance #0 (driver name: zvcntrl)zvterm, instance #0 (driver name: zvterm)zvblk, instance #0 (driver name: zvblk)zvnet, instance #0 (driver name: zvnet)zvsdir, instance #0 (driver name: zvsdir)fcoe, instance #0 (driver name: fcoe)iscsi, instance #0 (driver name: iscsi)options, instance #0 (driver name: options)pseudo, instance #0 (driver name: pseudo)vga_arbiter, instance #0 (driver name: vga_arbiter)xsvc, instance #0 (driver name: xsvc)cpus, instance #0 (driver name: cpunex)cpu (driver name: cpudrv) 45. SAS 40 CPU/ I/O SAS IO 4503:0002:3002:0001:3001:0000:3000:00Time (hh:mm)Bare Metal Kernel ZoneSAS9.4hNp://www.oracle.com/technetwork/database/bi-datawarehousing/sas/sas-on-sol11-2-kernelzones-2195675.pdf 46. Solaris Kernel Zones Native Zones Solaris Zones HW CPU HW 2GB ,(zvol, LUN) suspend/resume Live Migration () 46 47. 47Solaris 11.2 / 48. 2 BE BE ZFS ZFS 48 OVF (Open Virtualization Format) OVF OVA (Open Virtualization Format Archive) 49. Native or Kernel Zones?49 50. NaUveorKernelZones? Kernel Zones Native Kernel Zone 50NaUveKernelMobilityFixedMemoryIndependentVersionsMoreIsolaUonResourceSharingZeroOverheadIOLightweightHighConsolidaUon 51. Oracle Solaris Zones or OVM Server for SPARC? Oracle Solaris Zones OVM for SPARC NUMA HA Solaris Cluster Split PCI bus I/O (detach/attach) OS (DB12c PDB )51 52. OVM for SPARC Solaris Zones OVM for SPARC Solaris Zone other 52Solaris 11 ZoneSolaris 11 ZoneSolaris 11 Zone Solaris 11 Zone Solaris 11 ZoneSolaris 11 Zone Solaris 11 ZoneSolaris 11 Zone Solaris 10 ZoneSolaris 11 Zone Solaris 11 ZoneSolaris 11 Zone Solaris 11 ZoneSolaris 11 Zone Solaris 11 ZoneSolaris 11 ZoneHR LDom Finance LDom Sales LDom 53. OpenStack Oracle Solaris ZonesSolaris Zones OpenStack Component Solaris Foundation Technology (Nova) Solaris Zones Solaris Kernel Zones Nova (Neutron) NIC/VLAN Crossbow SDN Elastic Virtual Switch (Cinder) ZFS COMSTAR iSCSI Cinder (Glance) Solaris (Unified Archive) Zone RAD (Remote Administrator Daemon) OpenStack OpenStack OpenStack Solaris SMF OpenStack Solaris 11 OpenStack 54. OpenStack Solaris 11.2 HorizonCloud ManagementCloud APIsNova Neutron CinderSwiftGlance Oracle Solaris Zones Elastic Virtual Switch ZFS File System Unified Archives 55. OpenStackmulU-nodeonSPARCT5-8(withOVMforSPARC)OracleConfidenUal55 56. OpenStackmulU-nodeonSPARCT5-8(withOVMforSPARC):56T5-8ControlDomainControl-node(Ldom)net0 net1Compute-node01(Ldom)net0 net1Compute-node02(Ldom)net0 net1evsExt_net10.134.67.240/29(VLAN200)Int_net192.168.10.0/24(VLAN201)VPVP VPVPVP VPZone01vnicZone03vnicZone04vnicZone02vnicZone05vnicZone06vnicuplinkportuplinkportuplinkportcinder Neutron horizonZFSrpoolZFSrpoolZFSrpoolnovanova novaZonerootZonerootManagementCatalyst4948trunk trunk trunkEvscontrollerEvsnodeEvsnode 57. 57create -bset brand=solaris-kzset autoboot=falseset autoshutdown=shutdownset hostid=0x4ebe2024add anet set lower-link=autoset configure-allowed-address=trueset link-protection=mac-nospoofset mac-address=autoset evs=switch01 set id=0endadd device set storage=dev:/dev/zvol/dsk/%{global-rootzpool}/VARSHARE/zones/%{zonename}/disk%{id}set bootpri=0set id=0endadd capped-memory set physical=2Gendadd virtual-cpu CPU set ncpus=8endadd suspend suspend ()set path=/system/zones/%{zonename}/suspendendadd keysource (susend )set raw={base64}yRC4nwxXki2cw8YvesvYkw==endKernel Zones iSCSILUNsetstorage=iscsi://192.168.100.201:3260/target.iqn.1986-03.com.sun:02:413a759d-6d29-6147-eddc-c986e374e20b,lun.4 58. : Physical Domains +Oracle VM for SPARCor/and Solaris Kernel Zonesor/andSolaris ZonesOVM for SPARC +or/and Solaris Kernel Zonesor/andSolaris ZonesOSPhysical Domains(M6-32/M5-32)CPU/Oracle VM forSPARC(SPARC T4/T5/M5/M6)H/WH/WSolaris Zones Solaris Kernel CPU/,H/WSolaris Zones H/WZones (SPARC T4/T5/M5/M6)H/W/OS 59. M6-32 32384CPU 3,0725932 60. Solaris Zones: Operating System Support for ConsolidatingCommercial Workloads Daniel Price and Andrew Tucker Sun Microsystems, Inc. https://www.usenix.org/legacy/event/lisa04/tech/full_papers/price/price.pdf Oracle Solaris 11.2 Information Library () http://docs.oracle.com/cd/E36784_01/index.html Introduction to Oracle Solaris Zones () http://docs.oracle.com/cd/E36784_01/html/E36848/index.htmlOracle Confidential 60 61. Oracle Solaris 11.2 VM Oracle Solaris 11.2 VM Downloadshttp://www.oracle.com/technetwork/server-storage/solaris11/downloads/vm-templates-2245495.html Oracle Solaris 11.1 VM for Oracle VM VirtualBox VirtualBox Oracle Solaris 11.2 Oracle Solaris 10 VM Downloadshttp://www.oracle.com/technetwork/server-storage/solaris11/downloads/virtual-machines-1355605.html Oracle Solaris 10 VM Template for Oracle VM VirtualBox VirtualBox Oracle Solaris 10 1/13 Oracle VM Template for Oracle Solaris 10 Zones for SPARC/x86 Oracle Solaris 10 Zones Oracle Solaris 11.1 VM for Oracle VM VirtualBox VirtulBox Solaris 11.1 Solaris 10 Zones Solaris 11.2