Skip to content

PowerMeterMPI ¤

PowerMeterMPI(
    project_name="test_project",
    output_filepath=None,
    config_file=None,
    output_format="csv",
    print_to_cli=True,
)

PowerMeter¤

Multi-node capable class for monitoring power consumption.

Attributes:

  • DATETIME_FORMAT (str) –

    The format for datetime objects.

  • DEFAULT_CONFIG_FILE (str) –

    The default configuration file name to use for wrapper.

  • DEFAULT_OUTPUT_FILEPATH (str) –

    The default output file path to save results.

  • LOGGING_FILE (str) –

    The filename for the experimentation logging file.

  • project_name (str) –

    Name of the experimentation project.

  • config_file (str) –

    Path to the configuration file for the wrapper initialisation.

  • output_filepath (str) –

    Path to the output file for results of profiling.

  • output_format (str) –

    Format for the output file (e.g. csv).

  • print_to_cli (bool) –

    Flag to print the result of measurement in Terminal at the end (default True).

  • power (PowerWrapper) –

    Instance of PowerWrapper class for power measurement.

  • used_package (str) –

    Name of the package of algorithm to profile during power measurement.

  • used_algorithm (str) –

    Name of the profiled algorithm for power measurement.

  • used_algorithm_description (str) –

    Description of the algorithm used during power measurement.

  • logging_filename (str) –

    Path to the logging file of experiment.

Methods:

  • measure_power –

    Decorator to measure power consumption during the execution of a function.

  • __set_used_arguments –

    Set the arguments used during power measurement.

  • __call__ –

    Set the arguments used during power measurement using a decorator syntax.

  • __enter__ –

    Enter method for context manager. Starts power measurement.

  • __exit__ –

    Exit method for context manager. Stops power measurement.

  • start_measure –

    Start measuring power consumption.

  • stop_measure –

    Stop measuring power consumption.

  • __record_data_to_file –

    Record power data to a file.

  • __log_records –

    Log recorded power data.

Initialize the PowerMeter instance. This initialization is done on every node of the system.

Parameters:

  • config_file (str, default: None ) –

    Path to the configuration file.

  • project_name (str, default: 'test_project' ) –

    Name of the project.

  • output_filepath (str, default: None ) –

    Path to the output file.

  • output_format (str, default: 'csv' ) –

    Format for the output file.

  • print_to_cli (bool, default: True ) –

    To print the result of measurement in Terminal at the end

Source code in ea2p/src/power_meter_mpi.py 
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(self, project_name="test_project", output_filepath=None, config_file=None, output_format="csv", print_to_cli=True):
    """
    Initialize the PowerMeter instance. This initialization is done on every node of the system.

    Parameters:
        config_file (str): Path to the configuration file.
        project_name (str): Name of the project.
        output_filepath (str): Path to the output file.
        output_format (str): Format for the output file.
        print_to_cli (bool): To print the result of measurement in Terminal at the end
    """
    self.project_name = project_name
    self.config_file = Path(config_file) if config_file else Path.cwd() / self.DEFAULT_CONFIG_FILE
    self.output_filepath = Path(output_filepath) if output_filepath else Path.cwd() / self.DEFAULT__OUTPUT_FILEPATH
    self.output_format = output_format
    self.print_to_cli = print_to_cli

    self.power = PowerWrapper(self.config_file)

    self.used_package = ""
    self.used_algorithm = ""
    self.used_algorithm_description = ""

    self.logging_filename = PACKAGE_PATH / self.LOGGING_FILE

    # Initialize MPI
    self.comm = MPI.COMM_WORLD
    self.rank = self.comm.Get_rank()
    self.size = self.comm.Get_size()

__call__ ¤

__call__(package, algorithm, algorithm_description='')

Set the arguments used during power measurement using a decorator syntax.

Parameters:

  • package (str) –

    Package name of the algorithm to profile.

  • algorithm (str) –

    Name of the algorithm to profile in the list of instruction of the decorated function.

  • algorithm_description (str, default: '' ) –

    Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

Source code in ea2p/src/power_meter_mpi.py 
134
135
136
137
138
139
140
141
142
143
144
145
def __call__(self, package, algorithm, algorithm_description=""):
    """
    Set the arguments used during power measurement using a decorator syntax.

    Parameters:
        package (str): Package name of the algorithm to profile.
        algorithm (str): Name of the algorithm to profile in the list of instruction of the decorated function.
        algorithm_description (str): Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

    """
    self.__set_used_arguments(package, algorithm, algorithm_description=algorithm_description)
    return self

__enter__ ¤

__enter__()

Enter method for context manager. Starts power measurement.

Source code in ea2p/src/power_meter_mpi.py 
147
148
149
150
151
152
153
154
155
def __enter__(self):
    """
    Enter method for context manager. Starts power measurement.
    """
    self.start_measure(
        self.used_package,
        self.used_algorithm,
        algorithm_description=self.used_algorithm_description,
    )

__exit__ ¤

__exit__(exit_type, value, traceback)

Exit method for context manager. Stops power measurement on a rank.

Source code in ea2p/src/power_meter_mpi.py 
157
158
159
160
161
def __exit__(self, exit_type, value, traceback):
    """
    Exit method for context manager. Stops power measurement on a rank.
    """
    self.stop_measure()

__log_records ¤

__log_records(
    recorded_power,
    algorithm="",
    package="",
    algorithm_description="",
)

Log recorded power data.

Parameters:

  • recorded_power (DataFrame) –

    Recorded power data.

  • package (str, default: '' ) –

    Package name of the algorithm to profile.

  • algorithm (str, default: '' ) –

    Name of the algorithm to profile in the list of instruction of the decorated function.

  • algorithm_description (str, default: '' ) –

    Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

Source code in ea2p/src/power_meter_mpi.py 
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
def __log_records(self, recorded_power, algorithm="", package="", algorithm_description=""):
    """
    Log recorded power data.

    Parameters:
        recorded_power (DataFrame): Recorded power data.
        package (str): Package name of the algorithm to profile.
        algorithm (str): Name of the algorithm to profile in the list of instruction of the decorated function.
        algorithm_description (str): Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

    """
    payload_prefix = {
        "Project Name": self.project_name,
        "Datetime": datetime.datetime.now().strftime(self.DATETIME_FORMAT)
    }
    payload_suffix = {
        "Package": package,
        "Algorithm": algorithm,
        "Algorithm's parameters": algorithm_description,
    }
    written = self.__record_data_to_file(
        pd.concat(
            [pd.DataFrame(payload_prefix, index=[0]), recorded_power, pd.DataFrame(payload_suffix, index=[0])],
            axis=1,
        )
    )
    LOGGER.info("Recorded into a file? %s", written)

__record_data_to_file ¤

__record_data_to_file(data)

Record power data to a file.

Parameters:

  • data (DataFrame) –

    Power data to be recorded.

Returns: True if recording is successful, False otherwise.

Source code in ea2p/src/power_meter_mpi.py 
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
def __record_data_to_file(self, data):
    """
    Record power data to a file.

    Parameters:
        data (DataFrame): Power data to be recorded.
    Returns:
        True if recording is successful, False otherwise.
    """
    try:
        if self.output_filepath.exists():
            data.to_csv(self.output_filepath, mode="a", index=False, header=False)
        else:
            data.to_csv(self.output_filepath, index=False)
        return True
    except Exception as e:
        LOGGER.error("Error during the CSV writing process: %s", str(e))
        LOGGER.error(traceback.format_exc())
        return False

__set_used_arguments ¤

__set_used_arguments(
    package, algorithm, algorithm_description=""
)

Set the arguments used during power measurement.

Parameters:

  • package (str) –

    Package name of the algorithm to profile.

  • algorithm (str) –

    Name of the algorithm to profile in the list of instruction of the decorated function.

  • algorithm_description (str, default: '' ) –

    Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

Source code in ea2p/src/power_meter_mpi.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
def __set_used_arguments(self, package, algorithm, algorithm_description=""):
    """
    Set the arguments used during power measurement.

    Parameters:
        package (str): Package name of the algorithm to profile.
        algorithm (str): Name of the algorithm to profile in the list of instruction of the decorated function.
        algorithm_description (str): Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

    """
    self.used_package = package
    self.used_algorithm = algorithm
    self.used_algorithm_description = algorithm_description

measure_power ¤

measure_power(package, algorithm, algorithm_description='')

Decorator to measure power consumption during the execution of a function.

Parameters:

  • package (str) –

    Package name of the algorithm to profile.

  • algorithm (str) –

    Name of the algorithm to profile in the list of instruction of the decorated function.

  • algorithm_description (str, default: '' ) –

    Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

Returns: Decorator function.

Source code in ea2p/src/power_meter_mpi.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
def measure_power(self, package, algorithm, algorithm_description=""):
    """
    Decorator to measure power consumption during the execution of a function.

    Parameters:
        package (str): Package name of the algorithm to profile.
        algorithm (str): Name of the algorithm to profile in the list of instruction of the decorated function.
        algorithm_description (str): Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).
    Returns:
        Decorator function.
    """
    if not algorithm or not package:
        raise SyntaxError(
            "Please input a description for the function you are trying "
            "to monitor. Pass in the algorithm and the package you are "
            "trying to monitor"
        )

    def decorator(func):
        def wrapper(*args, **kwargs):
            self.start_measure(
                package,
                algorithm,
                algorithm_description=algorithm_description,
            )
            try:
                results = func(*args, **kwargs)
            finally:
                self.stop_measure()
            return results

        return wrapper

    return decorator

start_measure ¤

start_measure(package, algorithm, algorithm_description='')

Start measuring power consumption on each MPI rank.

Parameters:

  • package (str) –

    Package name of the algorithm to profile.

  • algorithm (str) –

    Name of the algorithm to profile in the list of instruction of the decorated function.

  • algorithm_description (str, default: '' ) –

    Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

Source code in ea2p/src/power_meter_mpi.py 
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
def start_measure(self, package, algorithm, algorithm_description=""):
    """
    Start measuring power consumption on each MPI rank. 

    Parameters:
        package (str): Package name of the algorithm to profile.
        algorithm (str): Name of the algorithm to profile in the list of instruction of the decorated function.
        algorithm_description (str): Description of the profiled algorithm acording to the experimental setup or tesbet details (eg, dataset used, epochs for training, batch size, etc...).

    """
    self.power.start()
    self.__set_used_arguments(
        package,
        algorithm,
        algorithm_description=algorithm_description,
    )

stop_measure ¤

stop_measure()

Stop measuring power consumption and gather results from all MPI processes.

Source code in ea2p/src/power_meter_mpi.py 
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
def stop_measure(self):
    """
    Stop measuring power consumption and gather results from all MPI processes.
    """
    self.power.stop()
    local_record = self.power.record

    # Add rank number to local record
    if local_record is not None:
        local_record['Rank'] = self.rank

    # Gathering data from all processes
    global_record = self.comm.gather(local_record, root=0)

    if self.rank == 0:
        # Filter out None values in case some processes had no data
        global_record = [record for record in global_record if record is not None]

        if global_record:
            # Concatenate data from all processes, preserving the original ranks
            global_record = pd.concat(global_record, ignore_index=True)

            # Compute total energy and add the 'Rank' column for the total row
            total_energy = global_record.sum(numeric_only=True)
            total_energy['Rank'] = 'Total'
            total_energy_row = pd.DataFrame([total_energy], columns=global_record.columns)
            total_energy_row = total_energy_row.round(5)

            # Append the total row to the global record
            global_record = pd.concat([global_record, total_energy_row], ignore_index=True)

            # Print the result if required
            if self.print_to_cli:
                print("Energy report for the experiment:\n")
                print(global_record)

            # Log the results
            self.__log_records(
                global_record,
                algorithm=self.used_algorithm,
                package=self.used_package,
                algorithm_description=self.used_algorithm_description,
            )
        else:
            print("No data was collected from the processes.")