status of florida tier2 center a mini tutorial on rocks appliances jorge l. rodriguez february 2003
TRANSCRIPT
![Page 1: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/1.jpg)
Status of Florida Tier2 Center
A mini tutorial on ROCKS appliances
Jorge L. Rodriguez
February 2003
![Page 2: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/2.jpg)
Future UF System Architecture
• Keep multiple head/pool of compute node architecture– Move user file systems from multiple headnodes and onto
grinhead• Simplify user maintenance• Implement this with NIS or LDAP directory services
– ROCKS is developing an LDAP implementation of user directory services for their cluster.
– Cluster management based on latest version of ROCKS • The grinhead node will be the ROCKS server• Define current configuration as appliance and modules• Integrate the new 3ware RAID fileserver as an appliance
• Integrate FNAL’s ROCKS based tier1/tier2 system– Must be able to handle local requirements, including GRID and
local projects of interests– Must be able to implement update scheme
From previous talk
![Page 3: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/3.jpg)
System Architecture
grinhead
pg igt dgt testhead others
grinux computational nodes
frontend node: (grinhead)User file systemROCKS database, dhcp, NIS, NFS server, Ganglia…
Head and Compute nodes etc (CE,WN, SE ): expressed as ROCKS appliances• Complete description including installation of VDT and local scheduler configuration• Good way of communicating installation procedures within the t1/t2 centers.
![Page 4: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/4.jpg)
Building ROCKS appliances• graphs and nodes
– nodes are collections of packages that provide function and include software and configurations
– graphs are collection of all nodes and edges defining a cluster
• Appliances are collection of nodes and edges that define a machine – left most node in a graph
(root)
A dot graph of the Florida cluster
![Page 5: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/5.jpg)
Appliances: graphs, nodes and xml
1. Modify the graph ( the default graph is in defaults.xml)Add statements in xml
/home/install/profiles/2.3/graphs/defaults.xml
<edge from="frontend-dgt" to="frontend-florida"/>
<edge from="frontend-dgt" to="vdt"/>
<edge from="frontend-dgt" to="VOMS"/>
<edge from="frontend-dgt" to="Mona-Lisa"/>
Note: ROCKS provides a site-nodes/ directory where one can insert nodes. It allows one to modify or replace existing nodes but I’m not sure exactly how to use it …
2. Add the nodes (.xml) files An appliance is just a node at the far left of the graph (root)
![Page 6: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/6.jpg)
Appliances: ROCKS Database3. Add the Appliance to the ROCKS database
1. Add an entry to the appliance tableid = primary key (auto increment)name = appliance name (e.g. frontend-igt)shortname = appliance short name (e.g. f-igt)graph = xml graph filename where appliance livesnode = root of graph (e.g. frontend-igt)
2. Add an entry to the memberships tableid = primary key (auto increment)name = human readable name (used by insert-
ethers)appliance = primary key from appliance table (id. appliance)distribution = primary key from distribution table compute = “yes” pbs jobs will be scheduled here “no”
they won’t
![Page 7: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/7.jpg)
Some useful commands
• Building the dot.gif files, for you viewing pleasurecd .../profiles/2.3 : On ROCKS frontend node
kpp [-g default-meet.xml ] | dot –Tgif > default-meet.gif– kpp is the kickstart pre processor, without argument it
produces a directed graph file that can be read by dot. dot draws the directed graph and outputs in various formats
• Building the kickstart filecd …/profiles/2.3
kpp <appliance name> | kgen > foo.kickstart– kpp with an argument generates more xml which is read
by kgen and translated to a kickstart file.
See section 4. in rocks doc “Leveraging Standard Core Technologies to Programmatically Build Linux Cluster Appliances”
![Page 8: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/8.jpg)
Installing the machine
The final step: install the appliance on a machine• Use the usual insert-ethers methods
1. Start up insert-ethers program on frontend node
2. Select the new appliance type from menu
3. Crank up the blank machine
• Mucking with tables (no power cycling)• modify the nodes table, replacing old appliance • insert-ethers –-update
• This regenerates all configurations files with new DB info
• shoot-node: reinstalls node as new appliance• This hasn’t been fully tested yet??
![Page 9: Status of Florida Tier2 Center A mini tutorial on ROCKS appliances Jorge L. Rodriguez February 2003](https://reader035.vdocuments.us/reader035/viewer/2022081811/56649f395503460f94c5627a/html5/thumbnails/9.jpg)
Status• Appliances defined
– headnodes (CE): Derived from slave-node-auto plus UF specific nodes + VDT installation + …
– computenodes (WN): Straight from ROCKS– fileservers (SE) : Derived from slave-node plus UF specific nodes
• Status: – WN are done, installation tested, good to go! – CE appliance has been installed on one machine, more xml needed– SE appliance defined need to write xml and test– Need to test installation without insert-ethers– Hope to finish before end of next week
• Major problems with new RAID fileserver– 3 ware + WD 180 GB drive problem!!!!– Sensitive to loose connectors? Giving them one last chance!