公平性指标 TensorBoard 插件示例 Colab

在 TensorFlow.org 上查看

在 Google Colab 中运行

在 GitHub 上查看

下载笔记本

概述

在本活动中，您将使用 TensorBoard 的公平性指标。使用该插件，您可以可视化运行的公平性评估，并轻松比较各组的性能。

导入

运行以下代码以安装所需的库。

pip install -q -U pip==20.2

pip install fairness_indicators 'absl-py<0.9,>=0.7'
pip install google-api-python-client==1.8.3
pip install tensorboard-plugin-fairness-indicators
pip install tensorflow-serving-api==2.9.0

重新启动运行时。 重新启动运行时后，继续执行以下单元格，无需再次运行之前的单元格。

# %tf.disable_v2_behavior() # Uncomment this line if running in Google Colab.

import datetime
import os
import tempfile
from tensorboard_plugin_fairness_indicators import summary_v2
import tensorflow.compat.v1 as tf

# example_model.py is provided in fairness_indicators package to train and
# evaluate an example model. 
from fairness_indicators import example_model

tf.compat.v1.enable_eager_execution()

2022-12-14 11:02:07.518601: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-12-14 11:02:07.518702: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-12-14 11:02:07.518712: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

数据和常量

# To know about dataset, check Fairness Indicators Example Colab at:
# https://github.com/tensorflow/fairness-indicators/blob/master/g3doc/tutorials/Fairness_Indicators_Example_Colab.ipynb

train_tf_file = tf.keras.utils.get_file('train.tf', 'https://storage.googleapis.com/civil_comments_dataset/train_tf_processed.tfrecord')
validate_tf_file = tf.keras.utils.get_file('validate.tf', 'https://storage.googleapis.com/civil_comments_dataset/validate_tf_processed.tfrecord')

BASE_DIR = tempfile.gettempdir()
TEXT_FEATURE = 'comment_text'
LABEL = 'toxicity'
FEATURE_MAP = {
    # Label:
    LABEL: tf.io.FixedLenFeature([], tf.float32),
    # Text:
    TEXT_FEATURE: tf.io.FixedLenFeature([], tf.string),

    # Identities:
    'sexual_orientation': tf.io.VarLenFeature(tf.string),
    'gender': tf.io.VarLenFeature(tf.string),
    'religion': tf.io.VarLenFeature(tf.string),
    'race': tf.io.VarLenFeature(tf.string),
    'disability': tf.io.VarLenFeature(tf.string),
}

Downloading data from https://storage.googleapis.com/civil_comments_dataset/train_tf_processed.tfrecord
488153424/488153424 [==============================] - 12s 0us/step
Downloading data from https://storage.googleapis.com/civil_comments_dataset/validate_tf_processed.tfrecord
324941336/324941336 [==============================] - 7s 0us/step

训练模型

model_dir = os.path.join(BASE_DIR, 'train',
                         datetime.datetime.now().strftime('%Y%m%d-%H%M%S'))

classifier = example_model.train_model(model_dir,
                                       train_tf_file,
                                       LABEL,
                                       TEXT_FEATURE,
                                       FEATURE_MAP)

INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_model_dir': '/tmpfs/tmp/train/20221214-110229', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': '/tmpfs/tmp/train/20221214-110229', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/training/training_util.py:396: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/training/training_util.py:396: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2022-12-14 11:02:29.418732: W tensorflow/core/common_runtime/graph_constructor.cc:1526] Importing a graph with a lower producer version 26 into an existing graph with producer version 1286. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/canned/head.py:399: NumericColumn._get_dense_tensor (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/canned/head.py:399: NumericColumn._get_dense_tensor (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/feature_column/feature_column.py:2188: NumericColumn._transform_feature (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/feature_column/feature_column.py:2188: NumericColumn._transform_feature (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.
Instructions for updating:
The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/training/adagrad.py:138: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/training/adagrad.py:138: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
2022-12-14 11:02:32.761164: W tensorflow/core/common_runtime/type_inference.cc:339] Type inference failed. This indicates an invalid graph that escaped type checking. Error message: INVALID_ARGUMENT: expected compatible input types, but input 1:
type_id: TFT_OPTIONAL
args {
  type_id: TFT_PRODUCT
  args {
    type_id: TFT_TENSOR
    args {
      type_id: TFT_INT64
    }
  }
}
 is neither a subtype nor a supertype of the combined inputs preceding it:
type_id: TFT_OPTIONAL
args {
  type_id: TFT_PRODUCT
  args {
    type_id: TFT_TENSOR
    args {
      type_id: TFT_INT32
    }
  }
}

    while inferring type of node 'dnn/zero_fraction/cond/output/_18'
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into /tmpfs/tmp/train/20221214-110229/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into /tmpfs/tmp/train/20221214-110229/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 59.342094, step = 0
INFO:tensorflow:loss = 59.342094, step = 0
INFO:tensorflow:global_step/sec: 106.458
INFO:tensorflow:global_step/sec: 106.458
INFO:tensorflow:loss = 56.15521, step = 100 (0.941 sec)
INFO:tensorflow:loss = 56.15521, step = 100 (0.941 sec)
INFO:tensorflow:global_step/sec: 119.044
INFO:tensorflow:global_step/sec: 119.044
INFO:tensorflow:loss = 47.173832, step = 200 (0.840 sec)
INFO:tensorflow:loss = 47.173832, step = 200 (0.840 sec)
INFO:tensorflow:global_step/sec: 121.513
INFO:tensorflow:global_step/sec: 121.513
INFO:tensorflow:loss = 55.923626, step = 300 (0.823 sec)
INFO:tensorflow:loss = 55.923626, step = 300 (0.823 sec)
INFO:tensorflow:global_step/sec: 117.954
INFO:tensorflow:global_step/sec: 117.954
INFO:tensorflow:loss = 55.986916, step = 400 (0.848 sec)
INFO:tensorflow:loss = 55.986916, step = 400 (0.848 sec)
INFO:tensorflow:global_step/sec: 117.137
INFO:tensorflow:global_step/sec: 117.137
INFO:tensorflow:loss = 41.569267, step = 500 (0.854 sec)
INFO:tensorflow:loss = 41.569267, step = 500 (0.854 sec)
INFO:tensorflow:global_step/sec: 119.858
INFO:tensorflow:global_step/sec: 119.858
INFO:tensorflow:loss = 45.4774, step = 600 (0.834 sec)
INFO:tensorflow:loss = 45.4774, step = 600 (0.834 sec)
INFO:tensorflow:global_step/sec: 109.664
INFO:tensorflow:global_step/sec: 109.664
INFO:tensorflow:loss = 51.01483, step = 700 (0.912 sec)
INFO:tensorflow:loss = 51.01483, step = 700 (0.912 sec)
INFO:tensorflow:global_step/sec: 116.795
INFO:tensorflow:global_step/sec: 116.795
INFO:tensorflow:loss = 47.566914, step = 800 (0.856 sec)
INFO:tensorflow:loss = 47.566914, step = 800 (0.856 sec)
INFO:tensorflow:global_step/sec: 120.663
INFO:tensorflow:global_step/sec: 120.663
INFO:tensorflow:loss = 47.934036, step = 900 (0.829 sec)
INFO:tensorflow:loss = 47.934036, step = 900 (0.829 sec)
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 1000...
INFO:tensorflow:Saving checkpoints for 1000 into /tmpfs/tmp/train/20221214-110229/model.ckpt.
INFO:tensorflow:Saving checkpoints for 1000 into /tmpfs/tmp/train/20221214-110229/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 1000...
INFO:tensorflow:Loss for final step: 51.091232.
INFO:tensorflow:Loss for final step: 51.091232.

使用公平性指标运行 TensorFlow 模型分析

此步骤可能需要 2 到 5 分钟。

tfma_eval_result_path = os.path.join(BASE_DIR, 'tfma_eval_result')

example_model.evaluate_model(classifier,
                             validate_tf_file,
                             tfma_eval_result_path,
                             'gender',
                             LABEL,
                             FEATURE_MAP)

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/eval_saved_model/encoding.py:132: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://tensorflowcn.cn/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/eval_saved_model/encoding.py:132: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://tensorflowcn.cn/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
2022-12-14 11:02:45.003050: W tensorflow/core/common_runtime/graph_constructor.cc:1526] Importing a graph with a lower producer version 26 into an existing graph with producer version 1286. Shape inference will have run different parts of the graph with different producer versions.
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/canned/head.py:635: auc (from tensorflow.python.ops.metrics_impl) is deprecated and will be removed in a future version.
Instructions for updating:
The value of AUC returned by this may race with the update so this is deprecated. Please use tf.keras.metrics.AUC instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_estimator/python/estimator/canned/head.py:635: auc (from tensorflow.python.ops.metrics_impl) is deprecated and will be removed in a future version.
Instructions for updating:
The value of AUC returned by this may race with the update so this is deprecated. Please use tf.keras.metrics.AUC instead.
WARNING:tensorflow:Trapezoidal rule is known to produce incorrect PR-AUCs; please switch to "careful_interpolation" instead.
WARNING:tensorflow:Trapezoidal rule is known to produce incorrect PR-AUCs; please switch to "careful_interpolation" instead.
WARNING:tensorflow:Trapezoidal rule is known to produce incorrect PR-AUCs; please switch to "careful_interpolation" instead.
WARNING:tensorflow:Trapezoidal rule is known to produce incorrect PR-AUCs; please switch to "careful_interpolation" instead.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/saved_model/model_utils/export_utils.py:84: get_tensor_from_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://tensorflowcn.cn/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow/python/saved_model/model_utils/export_utils.py:84: get_tensor_from_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://tensorflowcn.cn/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval']
INFO:tensorflow:Signatures INCLUDED in export for Eval: ['eval']
WARNING:tensorflow:Export includes no default signature!
WARNING:tensorflow:Export includes no default signature!
INFO:tensorflow:Restoring parameters from /tmpfs/tmp/train/20221214-110229/model.ckpt-1000
INFO:tensorflow:Restoring parameters from /tmpfs/tmp/train/20221214-110229/model.ckpt-1000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:Assets written to: /tmpfs/tmp/tfma_eval_model/temp-1671015764/assets
INFO:tensorflow:Assets written to: /tmpfs/tmp/tfma_eval_model/temp-1671015764/assets
INFO:tensorflow:SavedModel written to: /tmpfs/tmp/tfma_eval_model/temp-1671015764/saved_model.pb
INFO:tensorflow:SavedModel written to: /tmpfs/tmp/tfma_eval_model/temp-1671015764/saved_model.pb
WARNING:absl:Tensorflow version (2.11.0) found. Note that TFMA support for TF 2.0 is currently in beta
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/eval_saved_model/load.py:163: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.saved_model.load` instead.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/eval_saved_model/load.py:163: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.saved_model.load` instead.
INFO:tensorflow:Restoring parameters from /tmpfs/tmp/tfma_eval_model/1671015764/variables/variables
INFO:tensorflow:Restoring parameters from /tmpfs/tmp/tfma_eval_model/1671015764/variables/variables
2022-12-14 11:02:49.417145: W tensorflow/c/c_api.cc:291] Operation '{name:'dnn/metrics/auc_precision_recall/false_positives/Assign' id:1194 op device:{requested: '', assigned: ''} def:{ { {node dnn/metrics/auc_precision_recall/false_positives/Assign} } = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](dnn/metrics/auc_precision_recall/false_positives, dnn/metrics/auc_precision_recall/false_positives/Initializer/zeros)} }' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
2022-12-14 11:02:49.766072: W tensorflow/c/c_api.cc:291] Operation '{name:'dnn/metrics/auc_precision_recall/false_positives/Assign' id:1194 op device:{requested: '', assigned: ''} def:{ { {node dnn/metrics/auc_precision_recall/false_positives/Assign} } = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](dnn/metrics/auc_precision_recall/false_positives, dnn/metrics/auc_precision_recall/false_positives/Initializer/zeros)} }' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
WARNING:apache_beam.io.tfrecordio:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.
WARNING:apache_beam.io.filebasedsink:Deleting 1 existing files in target path matching:
WARNING:apache_beam.io.filebasedsink:Deleting 1 existing files in target path matching: -*-of-%(num_shards)05d
WARNING:apache_beam.io.filebasedsink:No shards found to finalize. num_shards: 1, skipped: 1
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/writers/metrics_plots_and_validations_writer.py:110: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.8/site-packages/tensorflow_model_analysis/writers/metrics_plots_and_validations_writer.py:110: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`

在 TensorBoard 中可视化公平性指标

您将在 Tensorboard 中可视化公平性指标，并比较数据每个切片在所选指标上的性能。您可以使用可视化顶部的下拉菜单调整基线比较切片以及显示的阈值。您还可以使用左上角的下拉菜单选择不同的评估运行。

编写公平性指标摘要

编写包含所有必要信息以在 TensorBoard 中可视化公平性指标的摘要文件。

import tensorflow.compat.v2 as tf2

writer = tf2.summary.create_file_writer(
    os.path.join(model_dir, 'fairness_indicators'))
with writer.as_default():
  summary_v2.FairnessIndicators(tfma_eval_result_path, step=1)
writer.close()

启动 TensorBoard

导航到“公平性指标”选项卡以可视化公平性指标。

%load_ext tensorboard

%tensorboard --logdir=$model_dir