6.0 Monitoring Metadata
- the user needs an estimation of how long it will take
to run a unit, a group unit, entire network on a particular machine.
- having started a job, the user wants to know the state
of the execution of the network at any time
- network monitor - bandwidth
- host monitoring (cpu, disk, memory, storage, javaflops, Triana
version, jdk, security)
- job monitoring progress, status (not started, started, stopped,
Currently, we are thinking of attacking this problem via two methods.
- Use the order of the algorithm directly or use
- self-learning, empirical estimation of the order of an algorithm
The self-learning, empirical estimation of the order of an algorithm
does not need to know the direction relationship between its
dimensionality and its timing. Calculate the FLOPS required for
the algorithm by using an empirical observation this can
be normalized using successive measurements. Also run the algorithm and
divide the time taken by the FLOPS.
T_ = T_ALG / (N*FLOPS)
T_ALG = Timing of algorithm (in seconds)
T_INDEX = Timing for unit data set on this machine
N = dimensionality of the data set (number of elements is data set for vectors)