	               Refactory of Install Program



	                      Li Zhi
	                      06/18/2010


1. Introduction
   Why I start to rewrite the install program? What problems are there in current install program? These are 2 questions being first raised. My answer is the current install program has 7 sins :-)
   1) Very verbose. Same or similar code being scattered everywhere.
   2) Complicated. I have asked the author why it is so complicated. He said it was the nature. I do not think so.
   3) No layer. There should be at least 3 layers: interface layer( GUI or curses), data layer, and operation layer. In current code, they are messed up, no layers seen.
   4) Different operating logic between different interfaces. GUI and curses used different operating code. This is very ridiculous. So it was always heard that bug XXX was fixed in GUI but not in curses, or vice versa.
   5) No unified data representation. It seems that every step stores its own information into one or more data files, with its own decided file names. It is very hard to have general, whole, deep insight on information used in install progress.
   6) Ridiculous representation on devices, partitions, etc. This is related to sin 5. It is strange and ridiculous to see that current install program can not handle HP machine, just because its hard disk is like "/dev/cciss/xxx", not like "/dev/sdaX".
   7) Monolithic. Although it is formed by several program files, you can not divide it easily. Can I hand write the data files, then run the install program? Can I write a script to run install program automatically? The answer is no.

   So, if you were I, would you re-write it or not?

2. Overview
   The key point, I deem as the first, is layering, with which sin 3 and 4 will be removed. I would like to divide the install program into 3 layers: interface layer, data manipulating layer, and install operation layer. 
   The second is to construct a unified data representtation, with which sin 5 and 6 will be removed.
   To sin 1 and 2, it is related to programming skills.
   If all above sins are removed, sin 7 will be probably distinguished too.

   The overall graph will be:


   Interface   	     	                  Data		     Operation

   ----------
  |          |
  |   GUI    |----------------
  |          |                \
   ----------                  \
                 -----------    \       -----------         -----------
                | XML for   |    \     |           |       |           |
                | Interface |     ---  | XML for   | ----- | Install   |
		 -----------     /     | Install   |       | Operation |
                                /      |           |       |           |
   ----------                  /        -----------         -----------
  |          |                /
  |  Curses  |----------------
  |          |
   ----------
                                 figure 2-1

3. Details
   Current install program is written in Tcl language. Tcl has a fame of not suitable to write big software project. And, I am not familar with it. So I choose Python as the programming language.

3.1 Interface layer
    There will be 2 interfaces: GUI and Curses. I want to have the same layout and the same style between them. So I need a central data file to record the widgets. That will be a "XML for Interface" file, as indicated in figure 2-1. In before, install program has 2 seperate interface, among which there's no relation. This is not good.

    One world, one dream!
    As mentioned above, there will be one unified xml file for both GUI and Curses. Another one I want to unify is the structure/relation among processes and windowes. In the current install program, there's a strange behavior. Every step starts a new process, which will display a window. For example, there are 3 steps: step1, step2, step3. In normal sequence, when you executes step 1, then step 2, and finally step 3, if you "ps" processes, you will find 3 processes. But when you execute step 1, step 2, then back to step 1, step 2 again, finally step 3, there will be 5 processes: step 1, step 2, step 1, step 2, and step 3. If you wander in steps back and forth more, you will get more processes. In GUI, you will find many windows too, for each process has a window.
    One method to solve this problem is to use "exec" rather than starting a new sub-process. Do you think that is best? OK, after I saw a demo program to show python plus dialog usage, I know there is a much better way. In my code, there will be only one process and only one window. What is changed is the display content among differnt steps.
    
3.1.1 GUI
      Python Tkinter is chosen as the tool. Grid is chose as the geometry manager.
      Basically, each window will be divided into 5x3 grids, 5 rows plus 3 columns. Among them, row 0, 1, 3, and 4 are same. The only changable one is row 2.
      The graph is as below:

        column 0            column 1            column 2
       ------------------------------------------------------------
row 0 | logo		   step name			version    |
row 1 |------------------------------------------------------------|
row 2 |                                                            |
row 3 | 		 copyright information	    		   |
row 4 | quit	    	      			 previous  next    |
       ------------------------------------------------------------
                            figure 3.1.1-1
      How to connect different windows? There are 2 alternatives. One is using Ids. Every window has a id, my program just sorts windows according to Ids. For example, there are 3 windows with id 1, 2, and 3. Number 1 window will be first displayed, when user clicks "next", number 2 window will be displayed. It seems reasonable, right? If it did not have the following 2 drawbacks, I would not think about the second alternative: first, id is an inborn attribute to each window, you have to decide the install sequence at the same time or before you define the window; second it is a little hard to solve the "detour" problem. The install sequence is not a straight line, but has some side road, which is called "detour" by I. For example, when in partition dividing, there may or may not be some windows for RAID setup.
      Now comes the second alternative: using names. I admit and believe that human beings like string while computers like number. Each window has a name. The install/setup sequence will be defined separately like ["welcome", "serial number", "partition dividing", "package selecting"]. For detour problem, another sequence will be defined, and some button in main sequence will open the detour sequence.

      I will just use top window to handle detour!

3.1.2 Curses
      I have investigated several choices: urwid, pycdk, and python+dialog. Some is a little complicated and error-prone: urwid. Some has few documents: pycdk. Finally, I chose python+dialog. It is simple, and workable. I like the demo.py in it very much. It demonstrates excellent programming skills.
      The curses interface will be simple and terse. It will only include the changing part in GUI interface(row 2).

3.1.3 XML and XML parser
      As previously mentioned, I will use XML to record widgets (Frame, Button, Label...). Here the detail will be discussed. For GUI, there shall be 3 kinds of information: Widgets attributes: background, height, width, text, command, etc, grid location - which row and column it will be located, and grid management. For details, please refer to:
      * http://www.pythonware.com/library/tkinter/
      * http://www.tkdocs.com/tutorial/
      Besides these, 2 things I also think about: multi-language support(at least English & Chinese) and inheritance among windows/frames. Can Chinese font be displayed in Curses mode? I'm not sure for now. And, at least supporting English is not a bad choice, I think. As I mentioned in section 3.1.1, most windows are similar, they are 5 rows X 3 columns, and only one row are different. So if I brought in a somehow inheritance way into xml format, it would save a lot of code lines and errors.
      To the first problem, I will design a data type "Message" in XML who has "key", "English text", and "Chinese text". All messages, such as a text on a button, will be references to "Message" variables, via key(for long message) or just English text(for short text). To the second problem, I first think about to introduce XML elements/attributes "derived_from" and "exception", which are similar to what I have done in building system implementation. But finally, I give up, for it is a little complex. Now it is enough to define a base widget for GUI row 0, 1, 3, and 4. While other widgets just define the content in row 2.

      group install: mandatory, yes, no
      service start: disable, yes, no

3.2 Data layer
    I have inspected those data files generated by current install program. After trying to merge them into one unified data, in xml format, I found it is unbelievable simple!

    Data layer will provide a xml file, and a series of functions to access data in it.

3.3 Operation
    This is a typical data-driven design. At last, based data read from xml file, operation layer does installing work. Roughly, it will include the following part:
    * partitioning
    * setting up raid(optional)
    * mounting
    * configuring network
    * packages installing
    * init scripts handling.
    * initrd generating
    * boot (grub/elilo/yaboot) installing

    Operation partition is very complicated. If I first record partition information in xml, then operate partitions based on data in xml, that will be difficult. For, first I have to record changes, some partitions may be untouched, some may be deleted, some may be changed a flag. If users operate partition list many times, how shall I do. Shall I record the action or the result? If action, order is critical. If result, I have to deduce actions. I do not like complexity. So I decided to let user use extern command like fdisk/parted directly.
    Ok, I know there's a side effect that partition will destroy data on disk. To minimize that, I will let partition operating be the last steps.

4. Hello Modules

   Nothing is particularly hard if you divide it into small jobs.
   - Henry Ford

   modules:


           .                            .                       .
           .    --------------------    .   -----------------   .
           .   |                    |   .  |                 |  .
           .   |  interface xml     |   .  |  install xml    |  .
           .   |                    |   .  |                 |  .
           .    --------------------    .   -----------------   .
           .                            .                       .
  -------  .    --------    ---------   .   --------            .  ---------
 | tk    | .   |  seq   |  | widget  |  .  |  xml   |           . | ???     |
  -------  .    --------    ---------   .   --------            .  ---------
           .                            .                       .
  -------- .    --------                .                       .
 | dialog |.   |  cmd   |               .                       .
  -------- .    --------                .                       .


   See, there're 4 layers. The newly added is for real window/Curses operatinng.

   * widget - mainly hold internal data structure for widget, messagebox, topwindow, etc. I can erase this module, just using xml representation. But that will be difficult to write code. And it will not be good to dynamically add (setattr)variable into xml node.

   * sequence - hold an internal data sequence.
   * cmd - all callback command, init command, quit command will be in this module.
   * tk - tk related operation
   * dialog - dialog related operation
   * xml    - xml parser
   * operation - install operation

   To support both Tk and dialog, I seperate display related functions from widget module into tk/dialog module. To connect different modules, I will:
   1) use call-back functions
   2) add init and quit functions to widget
   3) 
