VMDIRAC DEVELOPMENT PROPOSAL Zhao Xianghu Oct 14, 2014

Configuration of Agent Only one submit pool for the agent VirtualMachineScheduler no specific configurations 2

Configuration of Sites Site instead of RunningPod Site could include different cloud endpoints and images CLOUD.IHEP.cn CE = openstack.ihep.ac.cn, opennebula.ihep.ac.cn CEs openstack.ihep.ac.cn Endpoint = nova-1.1-ihep Images SL6-BOSS VO = bes Requirements = … Context = StandardCloudInit vmPolicy = never maxEndpointInstances = 20 SL5-CEPC VO = cepc Context = CEPCCloudInit cvmfs_http_proxy = DIRECT opennebula.ihep.ac.cn … 3

Configuration of VirtualMachine VirtualMachines CloudEndpoints nova-1.1-ihep … Images SL65-cvmfs-cloudinit … Context StandardCloudInit … 4

Configuration of Images Remove context Could be shared among different sites, if they have the same image name on cloud manager VirtualMachines Images SL65-cvmfs-cloudinit bootImageName = SL65-BOSS flavorName = m1.small 5

Configuration of Endpoints Only include configurations about the cloud Remove vmPolicy, vmStopPolicy, siteName, maxEndpointInstances, maxOportunisticEndpointInstances, cvmfs_http_proxy Could share among different sites VirtualMachines CloudEndpoints nova-1.1-ihep URI = auth = userpasswd … 6

Configuration of CloudInit Minimize the number of downloading scripts from internet Try to transfer most through context VirtualMachines Context StandardCloudInit Method = cloudinit CVMFSContext = … DiracContext = … 7

Current Structure 8 CloudDirector VMDirector VM Scheduler Image (Context) Client Context should be independent Image should not call client and should be simplified It is also actually not client. Better put in resources Director is not neccessary

Driver Interfaces check_connection(self, timelife = 10) create_VMInstance(self) getStatus_VMInstance( self, instanceId ) terminate_VMinstance( self, instanceId ) Following not present in current VMDIRAC get_cloud_status get_images get_flavors get_instances Need know more information about different cloud 9

Unify the Context Script Context Configure the VM (CVMFS, …) Install DIRAC and run agents Should be like the pilot Use generic_pilot proxy instead of host cert Use base64 to transfer scripts with single file Pilot itself do not concern about how it is started Could run continuously, different as normal pilot 10

Compared with BOINC BOINC does not have its own scheduler or site director Only a new CE type DIRAC.Resources.Computing.BOINCComputingElement Derived from DIRAC.Resources.Computing.ComputingElement A CE should implement the following: submitJob( self, executableFile, proxy, dummy = None ) getCEStatus( self ) getJobStatus( self, jobIDList ) getJobOutput( self, jobID, localDir = None ) 11

Add another CEType? Difficult to cooperate with SiteDirector No CE? No queue? No VM management? 12

Monitoring More detailed status submitted, deleted, context, running, stalled, halted, zombie Status should be collected from: Cloud manager VM monitor agent Controlling the VMs Need authority control 13

Log Save on a shared file system? Send by job? 14