report.html

Report generated on 27-Oct-2020 at 14:09:59 by pytest-html v2.0.1

Environment

CI true
CI_COMMIT_REF_NAME felix_issue199-percentiles-in-latex-report
CI_COMMIT_REF_SLUG felix-issue199-percentiles-in-latex-report
CI_COMMIT_SHA ac2aaed53f99d671d4fad66361497a6fb14895c9
CI_JOB_ID 230013
CI_JOB_NAME tests
CI_JOB_STAGE test
CI_JOB_TOKEN W2BHX5S2WLjktfqLHQs9
CI_PIPELINE_ID 49961
CI_PROJECT_DIR /builds/toar/mlair
CI_PROJECT_ID 2411
CI_PROJECT_NAME mlair
CI_PROJECT_NAMESPACE toar
CI_PROJECT_PATH toar/mlair
CI_PROJECT_URL https://gitlab.version.fz-juelich.de/toar/mlair
CI_REGISTRY gitlab.version.fz-juelich.de:5555
CI_REGISTRY_IMAGE gitlab.version.fz-juelich.de:5555/toar/mlair
CI_REGISTRY_PASSWORD W2BHX5S2WLjktfqLHQs9
CI_REGISTRY_USER gitlab-ci-token
CI_REPOSITORY_URL https://gitlab-ci-token:W2BHX5S2WLjktfqLHQs9@gitlab.version.fz-juelich.de/toar/mlair.git
CI_RUNNER_DESCRIPTION gitlab-runner on zam347 with already installed requirements of mlt using opensuse/leap
CI_RUNNER_ID 369
CI_RUNNER_TAGS mlt, machinelearningtools, leap, opensuse, zam347
CI_SERVER yes
CI_SERVER_NAME GitLab
CI_SERVER_REVISION e7d9e8b4224
CI_SERVER_VERSION 13.4.4
GITLAB_CI true
GITLAB_USER_EMAIL f.kleinert@fz-juelich.de
GITLAB_USER_ID 604
Packages {'pytest': '5.3.5', 'py': '1.8.1', 'pluggy': '0.13.1'}
Platform Linux-4.15.0-91-generic-x86_64-with-glibc2.3.4
Plugins {'lazy-fixture': '0.6.3', 'sugar': '0.9.2', 'html': '2.0.1', 'cov': '2.8.1', 'metadata': '1.8.0'}
Python 3.6.5

Summary

401 tests ran in 234.29 seconds.

401 passed, 0 skipped, 0 failed, 0 errors, 0 expected failures, 0 unexpected passes

Results

Result Test Duration Links
Passed test/test_datastore.py::TestAbstractDataStore::test_init 0.00
No log output captured.
Passed test/test_datastore.py::TestAbstractDataStore::test_clear_data_store 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_put 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_get 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_get_with_sub_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_get_with_not_existing_sub_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_raise_not_in_data_store 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_get_default 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_raise_not_in_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_list_all_scopes 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search_empty_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_list_all_names 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search_scope_and_all_superiors 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search_scope_return_all 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_search_scope_and_all_superiors_return_all 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_create_args_dict_default_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_create_args_dict_given_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_create_args_dict_missing_entry 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_set_args_from_dict 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByVariable::test_no_scope_given 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_put_with_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_get 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_get_with_sub_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_get_with_not_existing_sub_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_raise_not_in_data_store 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_get_default 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_raise_not_in_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_list_all_scopes 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search_empty_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_list_all_names 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search_scope_and_all_superiors 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search_scope_return_all 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_search_scope_and_all_superiors_return_all 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_create_args_dict_default_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_create_args_dict_given_scope 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_create_args_dict_missing_entry 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_set_args_from_dict 0.00
No log output captured.
Passed test/test_datastore.py::TestDataStoreByScope::test_no_scope_given 0.00
No log output captured.
Passed test/test_datastore.py::TestCorrectScope::test_init 0.00
No log output captured.
Passed test/test_helpers.py::TestToList::test_to_list 0.00
No log output captured.
Passed test/test_helpers.py::TestCheckPath::test_check_path_and_create 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:33 Created path: data/test DEBUG root:helpers.py:35 Path already exists: data/test
Passed test/test_helpers.py::TestLoss::test_l_p_loss 0.33
------------------------------Captured stdout call------------------------------
Epoch 1/1 4/4 [==============================] - 0s 23ms/step - loss: 1.2500 Epoch 1/1 4/4 [==============================] - 0s 9ms/step - loss: 2.2500
------------------------------Captured stderr call------------------------------
WARNING:tensorflow:From /usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. 2020-10-27 14:06:10,865 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. [deprecation.py:new_func:323] 2020-10-27 14:06:10.985995: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA 2020-10-27 14:06:11.020288: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2100000000 Hz 2020-10-27 14:06:11.023874: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x560ee665e3d0 executing computations on platform Host. Devices: 2020-10-27 14:06:11.023974: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined>
-------------------------------Captured log call--------------------------------
WARNING tensorflow:deprecation.py:323 From /usr/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer.
Passed test/test_helpers.py::TestTimeTracking::test_init 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test__start 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test__end 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test__duration 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test_repr 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test_run 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test_stop 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test_duration 0.00
No log output captured.
Passed test/test_helpers.py::TestTimeTracking::test_enter_exit 0.00
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:11,190 - INFO: undefined job finished after 0:00:01 (hh:mm:ss) [helpers.py:__exit__:113]
-------------------------------Captured log call--------------------------------
INFO root:helpers.py:113 undefined job finished after 0:00:01 (hh:mm:ss)
Passed test/test_helpers.py::TestTimeTracking::test_name_enter_exit 0.00
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:11,193 - INFO: my job finished after 0:00:01 (hh:mm:ss) [helpers.py:__exit__:113]
-------------------------------Captured log call--------------------------------
INFO root:helpers.py:113 my job finished after 0:00:01 (hh:mm:ss)
Passed test/test_helpers.py::TestGetHost::test_get_host 0.00
No log output captured.
Passed test/test_helpers.py::TestPrepareHost::test_prepare_host 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:149 set path to: /home/testUser/machinelearningtools/data/toar_daily/ DEBUG root:helpers.py:149 set path to: /home/testUser/Data/toar_daily/ DEBUG root:helpers.py:149 set path to: /home/testUser/Data/toar_daily/ DEBUG root:helpers.py:149 set path to: /p/project/cjjsc42/testUser/DATA/toar_daily/ DEBUG root:helpers.py:149 set path to: /p/project/deepacf/intelliaq/testUser/DATA/toar_daily/ DEBUG root:helpers.py:149 set path to: /home/testUser/machinelearningtools/data/toar_daily/
Passed test/test_helpers.py::TestPrepareHost::test_error_handling_unknown_host 0.00
No log output captured.
Passed test/test_helpers.py::TestPrepareHost::test_error_handling 0.00
No log output captured.
Passed test/test_helpers.py::TestPrepareHost::test_os_path_exists 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:33 Created path: /home/testUser/machinelearningtools/data/toar_daily/
Passed test/test_helpers.py::TestSetExperimentName::test_set_experiment 0.00
No log output captured.
Passed test/test_helpers.py::TestSetExperimentName::test_set_experiment_from_sys 0.00
No log output captured.
Passed test/test_helpers.py::TestSetExperimentName::test_set_expperiment_hourly 0.00
No log output captured.
Passed test/test_helpers.py::TestSetBootstrapPath::test_bootstrap_path_is_none 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:33 Created path: TestDataPath/../bootstrap_daily
Passed test/test_helpers.py::TestSetBootstrapPath::test_bootstap_path_is_given 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:33 Created path: Test/path/to/boots
Passed test/test_helpers.py::TestPytestRegex::test_pytest_regex_init 0.00
No log output captured.
Passed test/test_helpers.py::TestPytestRegex::test_pytest_regex_eq 0.00
No log output captured.
Passed test/test_helpers.py::TestPytestRegex::test_pytest_regex_repr 0.00
No log output captured.
Passed test/test_helpers.py::TestDictToXarray::test_dict_to_xarray 0.01
No log output captured.
Passed test/test_helpers.py::TestFloatRound::test_float_round_ceil 0.00
No log output captured.
Passed test/test_helpers.py::TestFloatRound::test_float_round_decimals 0.00
No log output captured.
Passed test/test_helpers.py::TestFloatRound::test_float_round_type 0.00
No log output captured.
Passed test/test_helpers.py::TestFloatRound::test_float_round_negative 0.00
No log output captured.
Passed test/test_helpers.py::TestDictPop::test_dict_pop_single 0.00
No log output captured.
Passed test/test_helpers.py::TestDictPop::test_dict_pop_multiple 0.00
No log output captured.
Passed test/test_helpers.py::TestDictPop::test_dict_pop_missing_argument 0.00
No log output captured.
Passed test/test_helpers.py::TestListPop::test_list_pop_single 0.00
No log output captured.
Passed test/test_helpers.py::TestListPop::test_list_pop_multiple 0.00
No log output captured.
Passed test/test_helpers.py::TestListPop::test_list_pop_missing_argument 0.00
No log output captured.
Passed test/test_helpers.py::TestLogger::test_init_default 0.00
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:11,265 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,265 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282]
-------------------------------Captured log call--------------------------------
INFO root:helpers.py:282 File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log
Passed test/test_helpers.py::TestLogger::test_setup_logging_path_none 0.00
No log output captured.
Passed test/test_helpers.py::TestLogger::test_setup_logging_path_given 0.00
No log output captured.
Passed test/test_helpers.py::TestLogger::test_logger_console_level0 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:06:11,274 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,274 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,274 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282]
-------------------------------Captured log setup-------------------------------
INFO root:helpers.py:282 File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log
Passed test/test_helpers.py::TestLogger::test_logger_console_level1 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:06:11,277 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,277 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,277 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,277 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282]
-------------------------------Captured log setup-------------------------------
INFO root:helpers.py:282 File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log
Passed test/test_helpers.py::TestLogger::test_logger_console_level_wrong_type 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:06:11,279 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,279 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,279 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,279 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282] 2020-10-27 14:06:11,279 - INFO: File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log [helpers.py:__init__:282]
-------------------------------Captured log setup-------------------------------
INFO root:helpers.py:282 File logger: /builds/toar/mlair/src/../logging/logging_2020-10-27_14-06-11.log
Passed test/test_join.py::TestJoinUrlBase::test_url 0.00
No log output captured.
Passed test/test_join.py::TestDownloadJoin::test_download_single_var 1.72
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:11,386 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:11,386 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:11,386 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:11,386 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:11,386 - INFO: load: o3 [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107 HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3
Passed test/test_join.py::TestDownloadJoin::test_download_empty 0.07
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=traffic HTTP/1.1" 200 None
Passed test/test_join.py::TestCorrectDataFormat::test_correct_data_format 0.00
No log output captured.
Passed test/test_join.py::TestGetData::test 0.08
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=UBA&parameter_name=o3,no2 HTTP/1.1" 200 None
Passed test/test_join.py::TestLoadSeriesInformation::test_standard_query 0.07
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107 HTTP/1.1" 200 None
Passed test/test_join.py::TestLoadSeriesInformation::test_empty_result 0.07
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=traffic HTTP/1.1" 200 None
Passed test/test_join.py::TestSaveToPandas::test_empty_df 0.01
No log output captured.
Passed test/test_join.py::TestSaveToPandas::test_not_empty_df 0.01
No log output captured.
Passed test/test_join.py::TestSaveToPandas::test_alternative_date_format 0.00
No log output captured.
Passed test/test_join.py::TestCorrectStatName::test_nothing_to_do 0.00
No log output captured.
Passed test/test_join.py::TestCorrectStatName::test_correct_string 0.00
No log output captured.
Passed test/test_join.py::TestLowerList::test_string_lowering 0.00
No log output captured.
Passed test/test_join.py::TestCreateUrl::test_minimal_args_given 0.00
No log output captured.
Passed test/test_join.py::TestCreateUrl::test_given_kwargs 0.00
No log output captured.
Passed test/test_join.py::TestCreateUrl::test_single_kwargs 0.00
No log output captured.
Passed test/test_join.py::TestCreateUrl::test_none_kwargs 0.00
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise[pandas-0] 0.02
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise[xarray-index] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise_inverse[pandas-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise_inverse[xarray-index] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_apply_standardise_inverse[pandas-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_apply_standardise_inverse[xarray-index] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise_apply[pandas-pd_mean-pd_std-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestStandardise::test_standardise_apply[xarray-xr_mean-xr_std-index] 0.00
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre[pandas-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre[xarray-index] 0.00
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre_inverse[pandas-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre_inverse[xarray-index] 0.00
No log output captured.
Passed test/test_statistics.py::TestCentre::test_apply_centre_inverse[pandas-0] 0.01
No log output captured.
Passed test/test_statistics.py::TestCentre::test_apply_centre_inverse[xarray-index] 0.00
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre_apply[pandas-pd_mean-0] 0.00
No log output captured.
Passed test/test_statistics.py::TestCentre::test_centre_apply[xarray-xr_mean-index] 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStrapGenerator::test_init 0.01
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:06:13,534 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:13,534 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:13,534 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:13,534 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:13,534 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:13,604 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:13,604 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:13,604 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:13,604 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:13,604 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:15,003 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:15,003 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:15,003 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:15,003 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:15,003 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log setup-------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStrapGenerator::test_len 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStrapGenerator::test_get_shuffled 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStrapGenerator::test_getitem 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
-------------------------------Captured log call--------------------------------
DEBUG root:bootstraps.py:42 boot: 0
Passed test/test_data_handling/test_bootstraps.py::TestBootStrapGenerator::test_next 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
-------------------------------Captured log call--------------------------------
DEBUG root:bootstraps.py:42 boot: 0 DEBUG root:bootstraps.py:42 boot: 1
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_init 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_create_shuffled_data_create_new 10.83
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:18,744 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:18,744 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:18,744 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:18,744 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:18,744 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:18,745 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:18,745 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:18,745 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:18,745 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:18,745 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:18,746 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:18,746 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:18,746 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:18,746 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:18,746 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:18,837 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:18,837 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:18,837 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:18,837 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:18,837 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:20,275 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:20,275 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:20,275 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:20,275 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:20,275 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:23,161 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:23,161 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:23,161 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:23,161 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:23,161 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:23,164 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:23,164 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:23,164 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:23,164 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:23,164 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:23,247 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:23,247 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:23,247 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:23,247 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:23,247 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:25,888 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:25,888 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:25,888 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:25,888 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:25,888 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:80 create bootstap data for DEBW107 WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:bootstraps.py:80 create bootstap data for DEBW013 WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_create_shuffled_data_some_valid 9.62
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:29,585 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:29,585 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:29,585 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:29,585 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:29,585 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:29,586 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:29,586 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:29,586 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:29,586 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:29,586 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:29,587 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:29,587 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:29,587 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:29,587 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:29,587 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:29,658 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:29,658 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:29,658 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:29,658 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:29,658 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:31,147 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:31,147 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:31,147 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:31,147 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:31,147 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:33,799 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:33,799 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:33,799 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:33,799 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:33,799 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:33,802 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:33,802 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:33,802 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:33,802 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:33,802 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:33,878 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:33,878 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:33,878 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:33,878 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:33,878 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:36,414 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:36,414 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:36,414 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:36,414 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:36,414 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:39,087 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,087 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,087 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,087 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,087 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,088 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,088 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,088 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,088 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,088 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80]
-------------------------------Captured log call--------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:80 create bootstap data for DEBW107 DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp_2010_2014_.pickle INFO root:bootstraps.py:80 create bootstap data for DEBW013 DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW013&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23010&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=88710&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp_2010_2014_.pickle INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:80 create bootstap data for DEBW013
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_set_file_path 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_valid_bootstrap_file_blank 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_valid_bootstrap_file_already_satisfied 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_valid_bootstrap_file_reload_data_window 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_valid_bootstrap_file_reload_data_boots 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_valid_bootstrap_file_reload_data_use_max_file_boot 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestCreateShuffledData::test_shuffle 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_init_no_shuffling 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_init_with_shuffling 12.14
------------------------------Captured stderr call------------------------------
2020-10-27 14:06:39,239 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,239 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,239 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,239 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,239 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:39,239 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,239 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,239 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,239 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,239 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:39,240 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:39,240 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:39,240 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:39,240 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:39,240 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:39,329 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:39,329 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:39,329 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:39,329 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:39,329 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:40,867 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:40,867 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:40,867 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:40,867 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:40,867 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:44,907 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:44,907 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:44,907 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:44,907 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:44,907 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:44,909 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:44,909 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:44,909 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:44,909 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:44,909 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:44,991 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:44,991 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:44,991 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:44,991 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:44,991 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:48,593 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:48,593 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:48,593 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:48,593 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:48,593 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:80 create bootstap data for DEBW107 WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:bootstraps.py:80 create bootstap data for DEBW013 WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_stations 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_variables 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_window_history_size 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_generator 0.05
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:06:51,398 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:51,398 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:51,398 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:51,398 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:51,398 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:06:51,399 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:51,399 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:51,399 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:51,399 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:51,399 - INFO: create bootstap data for DEBW107 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:51,400 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:51,400 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:51,400 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:51,400 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:51,400 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:51,482 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:51,482 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:51,482 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:51,482 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:51,482 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:52,917 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:52,917 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:52,917 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:52,917 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:52,917 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:55,592 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:55,592 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:55,592 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:55,592 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:55,592 - INFO: create bootstap data for DEBW013 [bootstraps.py:create_shuffled_data:80] 2020-10-27 14:06:55,595 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:55,595 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:55,595 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:55,595 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:55,595 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:06:55,680 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:55,680 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:55,680 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:55,680 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:55,680 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:06:58,136 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:58,136 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:58,136 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:58,136 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:06:58,136 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log setup-------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:80 create bootstap data for DEBW107 DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle INFO root:bootstraps.py:80 create bootstap data for DEBW013 DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW013&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23010&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=88710&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_2010_2014_.pickle
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_generator_different_generator 0.27
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:00,882 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74]
-------------------------------Captured log call--------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data INFO root:bootstraps.py:74 create / check shuffled bootstrap data DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_labels 0.01
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:01,158 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,158 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,158 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,158 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,158 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74]
-------------------------------Captured log setup-------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_orig_prediction 0.02
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:01,170 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,170 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,170 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,170 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,170 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74]
-------------------------------Captured log setup-------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_load_shuffled_data 0.19
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:01,194 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,194 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,194 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,194 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,194 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74]
-------------------------------Captured log setup-------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_shuffled_data_file 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:01,391 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,391 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,391 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,391 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74] 2020-10-27 14:07:01,391 - INFO: create / check shuffled bootstrap data [bootstraps.py:create_shuffled_data:74]
-------------------------------Captured log setup-------------------------------
INFO root:bootstraps.py:74 create / check shuffled bootstrap data
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_get_shuffled_data_file_not_found 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_create_file_regex 0.00
No log output captured.
Passed test/test_data_handling/test_bootstraps.py::TestBootStraps::test_filter_files 0.03
No log output captured.
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_init_defaults 0.00
-----------------------------Captured stderr setup------------------------------
WARNING:tensorflow:From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. 2020-10-27 14:07:01,656 - WARNING: From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. [deprecation.py:new_func:506] 2020-10-27 14:07:01,656 - WARNING: From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. [deprecation.py:new_func:506] 2020-10-27 14:07:01,656 - WARNING: From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. [deprecation.py:new_func:506] 2020-10-27 14:07:01,656 - WARNING: From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. [deprecation.py:new_func:506] 2020-10-27 14:07:01,656 - WARNING: From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`. [deprecation.py:new_func:506]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> WARNING tensorflow:deprecation.py:506 From /usr/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_get_model_rank 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_get_number_of_mini_batches 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_distribute_on_batches_single_loop 9.58
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:03,341 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:03,341 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:03,341 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:03,341 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:03,341 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:03,420 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:03,420 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:03,420 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:03,420 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:03,420 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:04,838 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:04,838 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:04,838 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:04,838 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:04,838 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:07,413 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:07,413 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:07,413 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:07,413 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:07,413 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:07,494 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:07,494 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:07,494 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:07,494 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:07,494 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:09,950 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:09,950 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:09,950 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:09,950 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:09,950 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW013&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23010&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=88710&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_distribute_on_batches_infinite_loop 0.52
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_len 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_len_two_stations 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW013_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_permute_data_no_permutation 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_permute_data 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_distribute_on_batches_upsampling_no_extremes_given 0.14
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_distributor.py::TestDistributor::test_distribute_on_batches_upsampling 0.29
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'>
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_init 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_repr 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_len 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_iter 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_next 4.08
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:16,330 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:16,330 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:16,330 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:16,330 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:16,330 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:16,408 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:16,408 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:16,408 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:16,408 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:16,408 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:17,859 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:17,859 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:17,859 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:17,859 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:17,859 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_getitem 0.01
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_setup_transformation_no_transformation 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_setup_transformation_calculate_statistics 5.46
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:20,446 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:20,446 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:20,446 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:20,446 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:20,446 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:20,512 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:20,512 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:20,512 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:20,512 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:20,512 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:22,984 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:22,984 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:22,984 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:22,984 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:22,984 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:25,431 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,431 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,431 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,431 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,431 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,796 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,796 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,796 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,796 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,796 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW013&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23010&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=88710&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_setup_transformation_use_given_statistics 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_setup_transformation_errors 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_accurate_transformation_standardise 0.31
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:25,957 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,957 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,957 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,957 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:25,957 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:26,096 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,096 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,096 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,096 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,096 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_accurate_transformation_centre 0.22
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:26,423 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,423 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,423 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,423 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,423 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:26,567 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,567 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,567 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,567 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,567 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_accurate_transformation_all_others 0.14
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:26,783 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,783 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,783 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,783 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,783 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_estimated_transformation_standardise 0.24
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:26,949 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,949 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,949 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,949 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:26,949 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:27,181 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,181 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,181 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,181 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,181 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_estimated_transformation_centre 0.17
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:27,385 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,385 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,385 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,385 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,385 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:27,559 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,559 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,559 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,559 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:27,559 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_calculate_estimated_transformation_all_others 0.03
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_get_station_key 0.00
No log output captured.
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_get_data_generator 0.30
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_get_data_generator_transform 0.10
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_get_data_generator_extremes 0.25
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_save_pickle_data 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_data_generator.py::TestDataGenerator::test_load_pickle_data 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_data_handling/data/tmp/DEBW107_o3_temp_2010_2014_.pickle
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_init 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:28,424 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:28,424 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:28,424 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:28,424 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:28,424 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:07:28,509 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:28,509 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:28,509 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:28,509 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:28,509 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:29,959 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:29,959 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:29,959 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:29,959 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:07:29,959 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_init_no_stats 0.00
No log output captured.
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_download_data 15.48
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:32,505 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:32,505 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:32,505 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:32,505 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:32,505 - INFO: load: o3 [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=UBA HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=17057&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_download_data_from_join 2.45
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:47,990 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:47,990 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:47,990 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:47,990 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:47,990 - INFO: load: o3 [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=UBA HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=17057&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_station_meta 2.46
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:50,443 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:50,443 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:50,443 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:50,443 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:50,443 - INFO: load: o3 [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=UBA HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=17057&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 DEBUG root:data_preparation.py:119 meta data does not agree with given request for station_type: traffic (requested) != background (local). Raise FileNotFoundError to trigger new grapping from web.
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_load_data_overwrite_local_data 6.43
------------------------------Captured stderr call------------------------------
2020-10-27 14:07:52,898 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:52,898 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:52,898 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:52,898 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:52,898 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:56,705 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:56,705 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:56,705 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:56,705 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:56,705 - INFO: load: o3 [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:85 overwrite_local_data is true, therefore reload /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc from JOIN DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=UBA HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=17057&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 DEBUG root:data_preparation.py:91 loaded new data from JOIN DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:85 overwrite_local_data is true, therefore reload /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc from JOIN DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=UBA HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=17057&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 DEBUG root:data_preparation.py:91 loaded new data from JOIN
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_load_data_keep_local_data 0.04
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_repr 0.00
No log output captured.
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_set_file_name_and_meta 0.00
No log output captured.
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_interpolate[opts0] 0.02
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:07:59,327 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:07:59,327 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:07:59,327 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:07:59,327 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:07:59,327 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:07:59,427 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:59,427 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:59,427 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:59,427 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:07:59,427 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:00,856 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:00,856 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:00,856 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:00,856 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:00,856 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:119 meta data does not agree with given request for network_name: AIRBASE (requested) != UBA (local). Raise FileNotFoundError to trigger new grapping from web. WARNING root:data_preparation.py:102 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_interpolate[opts1] 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_interpolate[opts2] 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_standardise 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_standardise_apply 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[10-3-standardise-] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[6-None-standardise-std, ] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[None-3-standardise-mean, ] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[19-None-centre-] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[None-2-centre-mean, ] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[8-2-centre-] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_inverse_transform_params[None-None-standardise-mean, std, ] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_centre 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_centre_apply 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_inverse[standardise] 0.03
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_inverse[centre] 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_errors[normalise] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_errors[unknownmethod] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_inverse_errors[normalise] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_transform_inverse_errors[unknownmethod] 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_get_transformation_information 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_remove_nan_no_hist_or_label 0.07
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_remove_nan 0.10
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_remove_nan_too_short 0.09
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_create_index_array 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_shift 0.07
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_make_history_window 0.06
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_make_labels 0.05
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_make_labels_multiple 0.03
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_make_observation 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_make_observation_multiple 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_slice 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_slice_prep 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_for_neg_concentrations 0.01
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_check_station 1.95
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:04,815 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:08:04,815 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:08:04,815 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:08:04,815 - WARNING: [data_preparation.py:load_data:102] 2020-10-27 14:08:04,815 - WARNING: [data_preparation.py:load_data:102]
-------------------------------Captured log call--------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:119 meta data does not agree with given request for station_type: traffic (requested) != background (local). Raise FileNotFoundError to trigger new grapping from web. WARNING root:data_preparation.py:102 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&station_type=traffic&network_name=dummy HTTP/1.1" 200 None
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_get_transposed_history 0.03
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_get_transposed_label 0.02
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes 0.06
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_from_list 0.11
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_wrong_extremes 0.04
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_right_tail 0.07
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_none_label 0.03
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
-------------------------------Captured log call--------------------------------
DEBUG root:data_preparation.py:453 ['DEBW107'] has `None' labels, skip multiply extremes
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_none_history 0.03
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
-------------------------------Captured log call--------------------------------
DEBUG root:data_preparation.py:453 ['DEBW107'] has `None' labels, skip multiply extremes
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_multiply_extremes_none_label_history 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
-------------------------------Captured log call--------------------------------
DEBUG root:data_preparation.py:453 ['DEBW107'] has `None' labels, skip multiply extremes
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_get_extremes_history 0.09
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_data_handling/test_data_preparation.py::TestDataPrep::test_get_extremes_label 0.09
-------------------------------Captured log setup-------------------------------
DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_data_handling/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_data_handling/data/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_get_padding_for_same_negative_kernel_size 0.00
------------------------------Captured stdout call------------------------------
In test_get_padding_for_same_negative_kernel_size
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_get_padding_for_same_strides_greater_one 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_get_padding_for_same_non_int_kernel 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_get_padding_for_same_stride_3d 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_get_padding_for_same_even_pad 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_negative_pads 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_len_of_pad_tuple 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_tuple_of_none_integer 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_tuple_of_tuple_none_integer_first 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_tuple_of_tuple_none_integer_second 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_valid_mix_of_int_and_tuple 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_invalid_mixed_tuple_and_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadUtils::test_check_padding_format_invalid_mixed_int_and_tuple 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_valid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_negative_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_invalid_format_float 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_invalid_format_string 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_tuple_of_valid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_tuple_of_invalid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_init_tuple_of_tuple_of_invalid_format 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestReflectionPadding2D::test_call 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_valid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_negative_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_invalid_format_float 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_invalid_format_string 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_tuple_of_valid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_tuple_of_invalid_int 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_init_tuple_of_tuple_of_invalid_format 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestSymmerticPadding2D::test_call 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_check_and_get_padding_zero_padding 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_check_and_get_padding_sym_padding 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_check_and_get_padding_ref_padding 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_check_and_get_padding_raises 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[SymPad2D] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[SymmetricPadding2D0] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[SymmetricPadding2D1] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[RefPad2D] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[ReflectionPadding2D0] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[ReflectionPadding2D1] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[ZeroPad2D] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[ZeroPadding2D0] 0.00
No log output captured.
Passed test/test_model_modules/test_advanced_paddings.py::TestPadding2D::test_call[ZeroPadding2D1] 0.00
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestGetActivation::test_string_act 0.00
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestGetActivation::test_sting_act_unknown 0.00
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestGetActivation::test_layer_act 0.00
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestGetActivation::test_layer_act_invalid 0.00
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestFlattenTail::test_flatten_tail_no_bound_no_regul_no_drop 0.03
No log output captured.
Passed test/test_model_modules/test_flatten_tail.py::TestFlattenTail::test_flatten_tail_all_settings 0.10
No log output captured.
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_block_part_name 0.00
No log output captured.
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_conv_tower_3x3 0.04
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_conv_tower_3x3_batch_norm 1.26
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_conv_tower_3x3_activation 0.09
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: tanh DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.LeakyReLU'>
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_conv_tower_1x1 0.02
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_conv_towers 0.08
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu DEBUG root:inception_model.py:58 Inception Block with activation: relu
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_create_pool_tower 0.03
No log output captured.
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_inception_block 0.24
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu DEBUG root:inception_model.py:58 Inception Block with activation: tanh DEBUG root:inception_model.py:58 Inception Block with activation: relu DEBUG root:inception_model.py:58 Inception Block with activation: tanh
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_inception_block_invalid_batchnorm 0.08
-------------------------------Captured log call--------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: relu DEBUG root:inception_model.py:58 Inception Block with activation: tanh
Passed test/test_model_modules/test_inception_model.py::TestInceptionModelBase::test_batch_normalisation 0.12
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestHistoryAdvanced::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestHistoryAdvanced::test_on_train_begin 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestLearningRateDecay::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestLearningRateDecay::test_check_param 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestLearningRateDecay::test_on_epoch_begin 1.23
------------------------------Captured stdout call------------------------------
Epoch 1/5 4/4 [==============================] - 1s 195ms/step - loss: 1.6106 Epoch 2/5 4/4 [==============================] - 0s 881us/step - loss: 1.5147 Epoch 3/5 4/4 [==============================] - 0s 624us/step - loss: 1.4220 Epoch 4/5 4/4 [==============================] - 0s 610us/step - loss: 1.3372 Epoch 5/5 4/4 [==============================] - 0s 746us/step - loss: 1.2555
------------------------------Captured stderr call------------------------------
WARNING:tensorflow:From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. 2020-10-27 14:08:09,900 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. [deprecation.py:new_func:323] 2020-10-27 14:08:09,900 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. [deprecation.py:new_func:323] 2020-10-27 14:08:09,900 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. [deprecation.py:new_func:323] 2020-10-27 14:08:09,900 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. [deprecation.py:new_func:323] 2020-10-27 14:08:09,900 - WARNING: From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. [deprecation.py:new_func:323] 2020-10-27 14:08:10,871 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:10,871 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:10,871 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:10,871 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:10,871 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,045 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,045 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,045 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,045 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,045 - INFO: Set learning rate to 0.02 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,048 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,048 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,048 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,048 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,048 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,051 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,051 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,051 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,051 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,051 - INFO: Set learning rate to 0.019 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,053 - INFO: Set learning rate to 0.01805 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,053 - INFO: Set learning rate to 0.01805 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,053 - INFO: Set learning rate to 0.01805 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,053 - INFO: Set learning rate to 0.01805 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:08:11,053 - INFO: Set learning rate to 0.01805 [keras_extensions.py:on_epoch_begin:90]
-------------------------------Captured log call--------------------------------
WARNING tensorflow:deprecation.py:323 From /usr/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. INFO root:keras_extensions.py:90 Set learning rate to 0.02 INFO root:keras_extensions.py:90 Set learning rate to 0.02 INFO root:keras_extensions.py:90 Set learning rate to 0.019 INFO root:keras_extensions.py:90 Set learning rate to 0.019 INFO root:keras_extensions.py:90 Set learning rate to 0.01805
Passed test/test_model_modules/test_keras_extensions.py::TestModelCheckpointAdvanced::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestModelCheckpointAdvanced::test_update_best 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestModelCheckpointAdvanced::test_update_callbacks 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestModelCheckpointAdvanced::test_on_epoch_end 0.00
------------------------------Captured stdout call------------------------------
Epoch 00001: val_loss did not improve from 6.00000 Epoch 00010: val_loss did not improve from 6.00000 Epoch 00011: val_loss improved from 6.00000 to 4.00000, saving model to ckpt.test Epoch 00011: save to /builds/toar/mlair/test/test_model_modules/callback_lr Epoch 00011: save to /builds/toar/mlair/test/test_model_modules/callback_hist Epoch 00011: saving model to ckpt.test Epoch 00011: save to /builds/toar/mlair/test/test_model_modules/callback_lr Epoch 00011: save to /builds/toar/mlair/test/test_model_modules/callback_hist
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_callbacks_set 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_callbacks_get 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_update_callback 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_add_callback 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_add_callback_raise 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_get_callbacks_as_dict 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_get_callbacks_no_dict 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_get_callback_by_name 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test__get_callbacks 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_get_checkpoint 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_create_model_checkpoint 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_load_callbacks 0.00
No log output captured.
Passed test/test_model_modules/test_keras_extensions.py::TestCallbackHandler::test_update_checkpoint 0.00
No log output captured.
Passed test/test_model_modules/test_linear_model.py::TestOrdinaryLeastSquareModel::test_constant_input_variable 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_init 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_model_property 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_all_empty 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_dict 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_attr 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_mix_attr_dict_no_duplicates 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_mix_attr_dict_valid_duplicates_optimizer 1.24
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_mix_attr_dict_valid_duplicates_none_optimizer 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_property_type_error 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_mix_attr_dict_invalid_duplicates_other_optimizer 0.03
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_mix_attr_dict_invalid_duplicates_same_optimizer_other_args 1.28
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compile_options_setter_as_dict_invalid_keys 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compare_keras_optimizers_equal 1.31
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compare_keras_optimizers_no_optimizer 0.01
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compare_keras_optimizers_other_parameters_run_sess 1.32
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_compare_keras_optimizers_other_parameters_none_sess 1.38
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_getattr 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_get_settings 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_custom_objects 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestAbstractModelClass::test_set_custom_objects 0.00
No log output captured.
Passed test/test_model_modules/test_model_class.py::TestMyPaperModel::test_init 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'>
Passed test/test_model_modules/test_model_class.py::TestMyPaperModel::test_set_model 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'>
Passed test/test_model_modules/test_model_class.py::TestMyPaperModel::test_set_compile_options 0.00
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'>
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_set_param_by_value 0.00
-------------------------------Captured log call--------------------------------
DEBUG root:experiment_setup.py:155 set experiment attribute: 23tester(general)=23
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_set_param_by_value_and_scope 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,842 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,842 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,842 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,842 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,842 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
DEBUG root:experiment_setup.py:155 set experiment attribute: 109tester(general)=109
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_set_param_with_default 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,844 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,844 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,844 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,844 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,844 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
DEBUG root:experiment_setup.py:155 set experiment attribute: NoneTester(general.testing)=notNone DEBUG root:experiment_setup.py:155 set experiment attribute: AnotherNoneTester(general)=None
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_get_parser_args_from_dict 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,847 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,847 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,847 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,847 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,847 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_get_parser_args_from_parse_args 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,849 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,849 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,849 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,849 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,849 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_init_default 0.01
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,853 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,853 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,853 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,853 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,853 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:21,854 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,854 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,854 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,854 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,854 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,865 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,865 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,865 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,865 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,865 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:33 Created path: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:33 Created path: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBY052', 'DEBY032', 'DEBW022', 'DEBY004', 'DEBY020', 'DEBW030', 'DEBW037', 'DEBW031', 'DEBW015', 'DEBW073', 'DEBY039', 'DEBW038', 'DEBW081', 'DEBY075', 'DEBW040', 'DEBY053', 'DEBW059', 'DEBW027', 'DEBY072', 'DEBW042', 'DEBW039', 'DEBY001', 'DEBY113', 'DEBY089', 'DEBW024', 'DEBW004', 'DEBY037', 'DEBW056', 'DEBW029', 'DEBY068', 'DEBW010', 'DEBW026', 'DEBY002', 'DEBY079', 'DEBW084', 'DEBY049', 'DEBY031', 'DEBW019', 'DEBW001', 'DEBY063', 'DEBY005', 'DEBW046', 'DEBW103', 'DEBW052', 'DEBW034', 'DEBY088'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum', 'u': 'average_values', 'v': 'average_values', 'no': 'dma8eu', 'no2': 'dma8eu', 'cloudcover': 'average_values', 'pblheight': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum', 'temp', 'u', 'v', 'no', 'no2', 'cloudcover', 'pblheight'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_init_no_default 0.01
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:21,867 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,867 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,867 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,867 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,867 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,877 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,877 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,877 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,877 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,877 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.5 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBY053', 'DEBW059', 'DEBW027'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=INTERNET DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1999-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2001-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=4 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=relhum DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=target DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=10 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)=dim1 DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=int_dim DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=cubic DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=5 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=2000-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2000-01-02 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2000-01-03 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2000-01-04 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2000-01-05 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2000-01-06 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=2000-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2000-01-04 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=110 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=False DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_init_trainable_behaviour 0.03
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:21,879 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,879 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,879 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,879 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,879 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,887 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,887 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,887 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,887 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,887 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,894 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,902 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,909 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,909 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,909 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,909 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,909 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,910 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,910 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,910 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,910 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,910 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBY052', 'DEBY032', 'DEBW022', 'DEBY004', 'DEBY020', 'DEBW030', 'DEBW037', 'DEBW031', 'DEBW015', 'DEBW073', 'DEBY039', 'DEBW038', 'DEBW081', 'DEBY075', 'DEBW040', 'DEBY053', 'DEBW059', 'DEBW027', 'DEBY072', 'DEBW042', 'DEBW039', 'DEBY001', 'DEBY113', 'DEBY089', 'DEBW024', 'DEBW004', 'DEBY037', 'DEBW056', 'DEBW029', 'DEBY068', 'DEBW010', 'DEBW026', 'DEBY002', 'DEBY079', 'DEBW084', 'DEBY049', 'DEBY031', 'DEBW019', 'DEBW001', 'DEBY063', 'DEBY005', 'DEBW046', 'DEBW103', 'DEBW052', 'DEBW034', 'DEBY088'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum', 'u': 'average_values', 'v': 'average_values', 'no': 'dma8eu', 'no2': 'dma8eu', 'cloudcover': 'average_values', 'pblheight': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum', 'temp', 'u', 'v', 'no', 'no2', 'cloudcover', 'pblheight'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=False DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=False DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBY052', 'DEBY032', 'DEBW022', 'DEBY004', 'DEBY020', 'DEBW030', 'DEBW037', 'DEBW031', 'DEBW015', 'DEBW073', 'DEBY039', 'DEBW038', 'DEBW081', 'DEBY075', 'DEBW040', 'DEBY053', 'DEBW059', 'DEBW027', 'DEBY072', 'DEBW042', 'DEBW039', 'DEBY001', 'DEBY113', 'DEBY089', 'DEBW024', 'DEBW004', 'DEBY037', 'DEBW056', 'DEBW029', 'DEBY068', 'DEBW010', 'DEBW026', 'DEBY002', 'DEBY079', 'DEBW084', 'DEBY049', 'DEBY031', 'DEBW019', 'DEBW001', 'DEBY063', 'DEBY005', 'DEBW046', 'DEBW103', 'DEBW052', 'DEBW034', 'DEBY088'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum', 'u': 'average_values', 'v': 'average_values', 'no': 'dma8eu', 'no2': 'dma8eu', 'cloudcover': 'average_values', 'pblheight': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum', 'temp', 'u', 'v', 'no', 'no2', 'cloudcover', 'pblheight'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBY052', 'DEBY032', 'DEBW022', 'DEBY004', 'DEBY020', 'DEBW030', 'DEBW037', 'DEBW031', 'DEBW015', 'DEBW073', 'DEBY039', 'DEBW038', 'DEBW081', 'DEBY075', 'DEBW040', 'DEBY053', 'DEBW059', 'DEBW027', 'DEBY072', 'DEBW042', 'DEBW039', 'DEBY001', 'DEBY113', 'DEBY089', 'DEBW024', 'DEBW004', 'DEBY037', 'DEBW056', 'DEBW029', 'DEBY068', 'DEBW010', 'DEBW026', 'DEBY002', 'DEBY079', 'DEBW084', 'DEBY049', 'DEBY031', 'DEBW019', 'DEBW001', 'DEBY063', 'DEBY005', 'DEBW046', 'DEBW103', 'DEBW052', 'DEBW034', 'DEBY088'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum', 'u': 'average_values', 'v': 'average_values', 'no': 'dma8eu', 'no2': 'dma8eu', 'cloudcover': 'average_values', 'pblheight': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum', 'temp', 'u', 'v', 'no', 'no2', 'cloudcover', 'pblheight'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=False DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBY052', 'DEBY032', 'DEBW022', 'DEBY004', 'DEBY020', 'DEBW030', 'DEBW037', 'DEBW031', 'DEBW015', 'DEBW073', 'DEBY039', 'DEBW038', 'DEBW081', 'DEBY075', 'DEBW040', 'DEBY053', 'DEBW059', 'DEBW027', 'DEBY072', 'DEBW042', 'DEBW039', 'DEBY001', 'DEBY113', 'DEBY089', 'DEBW024', 'DEBW004', 'DEBY037', 'DEBW056', 'DEBW029', 'DEBY068', 'DEBW010', 'DEBW026', 'DEBY002', 'DEBY079', 'DEBW084', 'DEBY049', 'DEBY031', 'DEBW019', 'DEBW001', 'DEBY063', 'DEBY005', 'DEBW046', 'DEBW103', 'DEBW052', 'DEBW034', 'DEBY088'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'relhum': 'average_values', 'temp': 'maximum', 'u': 'average_values', 'v': 'average_values', 'no': 'dma8eu', 'no2': 'dma8eu', 'cloudcover': 'average_values', 'pblheight': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum', 'temp', 'u', 'v', 'no', 'no2', 'cloudcover', 'pblheight'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_experiment_setup.py::TestExperimentSetup::test_compare_variables_and_statistics 0.01
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:21,912 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,912 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,912 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,912 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,912 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,916 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,916 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,916 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,916 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,916 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,923 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,923 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,923 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,923 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,923 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBY053', 'DEBW059', 'DEBW027'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'relhum'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/data/testExperimentFolder/TODAY_network/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBY053', 'DEBW059', 'DEBW027'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=None DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_set_callbacks 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,926 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,926 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,926 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,926 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,926 - INFO: ModelSetup started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ModelSetup started
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_set_callbacks_no_lr_decay 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,929 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,929 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,929 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,929 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,929 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,930 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,930 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,930 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,930 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,930 - INFO: ModelSetup started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ModelSetup started
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_get_model_settings 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,932 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,932 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,932 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,932 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,932 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,933 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,933 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,933 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,933 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,933 - INFO: ModelSetup started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ModelSetup started
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_build_model 1.51
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:21,937 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,937 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,937 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,937 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,937 - INFO: ModelSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,938 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,938 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,938 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,938 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:21,938 - INFO: ModelSetup started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ModelSetup started
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:21,941 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,941 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,941 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,941 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:21,941 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.ELU'>
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_set_channels 5.88
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:23,455 - INFO: ModelSetup finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:23,455 - INFO: ModelSetup finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:23,455 - INFO: ModelSetup finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:23,455 - INFO: ModelSetup finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:23,455 - INFO: ModelSetup finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:23,456 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:23,456 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:23,456 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:23,456 - INFO: ModelSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:23,456 - INFO: ModelSetup started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ModelSetup started
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:23,459 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:23,459 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:23,459 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:23,459 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:23,459 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:25,345 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:25,345 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:25,345 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:25,345 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:25,345 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:26,785 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:26,785 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:26,785 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:26,785 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:26,785 - INFO: load: temp [join.py:download_join:59]
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_modules/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_load_weights 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:29,337 - INFO: ModelSetup finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,337 - INFO: ModelSetup finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,337 - INFO: ModelSetup finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,337 - INFO: ModelSetup finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,337 - INFO: ModelSetup finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39]
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_compile_model 0.00
No log output captured.
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_run 0.00
No log output captured.
Passed test/test_modules/test_model_setup.py::TestModelSetup::test_init 0.00
No log output captured.
Passed test/test_modules/test_partition_check.py::TestPartitionCheck::test_init 0.01
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:29,348 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,348 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,348 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,348 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,348 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,357 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,357 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,357 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,357 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,357 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,358 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:08:29,358 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:08:29,359 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,359 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,359 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,359 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,359 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,359 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:08:29,359 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:08:29,359 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:08:29,359 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:08:29,359 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 PartitionCheck started INFO root:run_environment.py:39 PartitionCheck finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:61 Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log
Passed test/test_modules/test_partition_check.py::TestPartitionCheck::test_run_login 0.01
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:29,366 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,366 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,366 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,366 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,366 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,376 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,376 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,376 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,376 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,376 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,376 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,376 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,376 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,376 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,376 - INFO: PartitionCheck started [run_environment.py:__init__:30]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:33 Created path: /p/project/deepacf/intelliaq/testUser/DATA/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/p/project/deepacf/intelliaq/testUser/DATA/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=juwels DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:33 Created path: /p/project/deepacf/intelliaq/testUser/DATA/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/p/project/deepacf/intelliaq/testUser/DATA/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PartitionCheck started
Passed test/test_modules/test_partition_check.py::TestPartitionCheck::test_run_compute 0.01
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:29,381 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,381 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,381 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,381 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,381 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,389 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,389 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,389 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,389 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,389 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,389 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,389 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,389 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,389 - INFO: PartitionCheck started [run_environment.py:__init__:30] 2020-10-27 14:08:29,389 - INFO: PartitionCheck started [run_environment.py:__init__:30]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:33 Created path: /p/project/deepacf/intelliaq/testUser/DATA/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/p/project/deepacf/intelliaq/testUser/DATA/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=hdfmlc01 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:33 Created path: /p/project/deepacf/intelliaq/testUser/DATA/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/p/project/deepacf/intelliaq/testUser/DATA/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:33 Created path: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PartitionCheck started
Passed test/test_modules/test_post_processing.py::TestPostProcessing::test_init 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:08:29,391 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,391 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,391 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,391 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,391 - INFO: PartitionCheck finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_init 33.46
------------------------------Captured stderr call------------------------------
2020-10-27 14:08:29,395 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,395 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,395 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,395 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,395 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:08:29,402 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,402 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,402 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,402 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,402 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:29,403 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:08:29,403 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:08:29,403 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:08:29,403 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:08:29,403 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:08:29,405 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:29,405 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:29,405 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:29,405 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:29,405 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:29,405 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:29,405 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:29,405 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:29,405 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:29,405 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:29,406 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:29,406 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:29,406 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:29,406 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:29,406 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:29,497 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:29,497 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:29,497 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:29,497 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:29,497 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:30,925 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:30,925 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:30,925 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:30,925 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:30,925 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:34,232 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:34,232 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:34,232 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:34,232 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:34,232 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:34,234 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:34,234 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:34,234 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:34,234 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:34,234 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:34,314 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:34,314 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:34,314 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:34,314 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:34,314 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:37,750 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:37,750 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:37,750 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:37,750 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:37,750 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:41,583 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:41,583 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:41,583 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:41,583 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:41,583 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:41,584 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:41,584 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:41,584 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:41,584 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:41,584 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:41,662 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:41,662 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:41,662 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:41,662 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:41,662 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:44,188 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:44,188 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:44,188 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:44,188 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:44,188 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:46,813 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:46,813 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:46,813 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:46,813 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:46,813 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:46,815 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:46,815 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:46,815 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:46,815 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:46,815 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:46,903 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:46,903 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:46,903 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:46,903 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:46,903 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:49,592 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:49,592 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:49,592 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:49,592 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:49,592 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:53,096 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:53,096 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:53,096 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:53,096 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:53,096 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:53,097 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:53,097 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:53,097 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:53,097 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:53,097 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:08:53,187 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:53,187 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:53,187 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:53,187 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:53,187 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:08:55,798 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:55,798 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:55,798 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:55,798 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:55,798 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:08:59,450 - INFO: run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:08:59,450 - INFO: run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:08:59,450 - INFO: run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:08:59,450 - INFO: run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:08:59,450 - INFO: run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:08:59,454 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:59,454 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:59,454 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:59,454 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:59,454 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:08:59,673 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,673 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,673 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,673 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,673 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,819 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,819 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,819 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,819 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,819 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,870 - INFO: PartitionCheck finished after 0:00:31 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:59,870 - INFO: PartitionCheck finished after 0:00:31 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:59,870 - INFO: PartitionCheck finished after 0:00:31 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:59,870 - INFO: PartitionCheck finished after 0:00:31 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:59,870 - INFO: PartitionCheck finished after 0:00:31 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:08:59,964 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,964 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,964 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,964 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:08:59,964 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,106 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,106 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,106 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,106 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,106 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,248 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,248 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,248 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,248 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,248 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,398 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:00,398 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:00,398 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:00,398 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:00,398 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:00,622 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:00,622 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:00,622 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:00,622 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:00,622 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:00,622 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,622 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,622 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,622 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,622 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,759 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,759 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,759 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,759 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,759 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,888 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,888 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,888 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,888 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:00,888 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,005 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,005 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,005 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,005 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,005 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,140 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,140 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,140 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,140 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,140 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,275 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,275 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,275 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,275 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,275 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,280 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,280 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,280 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,280 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,280 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,280 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,280 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,280 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,280 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,280 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,419 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,419 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,419 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,419 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,419 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,556 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,556 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,556 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,556 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,556 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,694 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,694 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,694 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,694 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,694 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,840 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,840 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,840 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,840 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,840 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,980 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,980 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,980 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,980 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,980 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:01,984 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,984 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,984 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,984 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,984 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:01,984 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,984 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,984 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,984 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:01,984 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,127 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,127 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,127 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,127 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,127 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,271 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,271 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,271 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,271 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,271 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,414 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,414 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,414 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,414 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,414 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,558 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,558 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,558 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,558 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,558 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,698 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:02,698 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:02,698 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:02,698 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:02,698 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:02,856 - INFO: PreProcessing finished after 0:00:34 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,856 - INFO: PreProcessing finished after 0:00:34 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,856 - INFO: PreProcessing finished after 0:00:34 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,856 - INFO: PreProcessing finished after 0:00:34 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,856 - INFO: PreProcessing finished after 0:00:34 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,856 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:02,856 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:02,856 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:02,856 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:02,856 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:02,857 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,857 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,857 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,857 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,857 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,857 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:02,857 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:02,857 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:02,857 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:02,857 - INFO: Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 PreProcessing started INFO root:pre_processing.py:189 check valid stations started (all) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc' INFO root:join.py:59 load: o3 INFO root:join.py:59 load: temp INFO root:pre_processing.py:207 run for 0:00:31 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. INFO root:pre_processing.py:189 check valid stations started (train) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) INFO root:run_environment.py:39 PartitionCheck finished after 0:00:31 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. INFO root:pre_processing.py:189 check valid stations started (val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. INFO root:pre_processing.py:189 check valid stations started (test) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. INFO root:pre_processing.py:189 check valid stations started (train_val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. INFO root:run_environment.py:39 PreProcessing finished after 0:00:34 (hh:mm:ss) INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:61 Copy log file to /builds/toar/mlair/TestExperiment/logging_000.log
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_run 4.19
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:02,862 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:02,862 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:02,862 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:02,862 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:02,862 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:02,870 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,870 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,870 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,870 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,870 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:02,870 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:02,870 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:02,870 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:02,870 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:02,870 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:02,873 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:02,873 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:02,873 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:02,873 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:02,873 - INFO: check valid stations started (all) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:02,874 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,874 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,874 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,874 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:02,874 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,006 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,006 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,006 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,006 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,006 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,150 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,150 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,150 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,150 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,150 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,289 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,289 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,289 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,289 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,289 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,427 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,427 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,427 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,427 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,427 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,567 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,567 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,567 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,567 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,567 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,568 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:03,568 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:03,568 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:03,568 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:03,568 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:03,654 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:03,654 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:03,654 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:03,654 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:03,654 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:03,662 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:03,662 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:03,662 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:03,662 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:03,662 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:03,922 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,922 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,922 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,922 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:03,922 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,066 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,066 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,066 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,066 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,066 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,210 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,210 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,210 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,210 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,210 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,364 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,364 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,364 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,364 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,364 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,527 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,527 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,527 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,527 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,527 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,672 - INFO: run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:04,672 - INFO: run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:04,672 - INFO: run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:04,672 - INFO: run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:04,672 - INFO: run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:04,883 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:04,883 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:04,883 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:04,883 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:04,883 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:04,884 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,884 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,884 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,884 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:04,884 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,020 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,020 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,020 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,020 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,020 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,157 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,157 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,157 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,157 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,157 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,294 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,294 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,294 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,294 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,294 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,432 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,432 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,432 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,432 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,432 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,562 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:05,562 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:05,562 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:05,562 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:05,562 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:05,566 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:05,566 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:05,566 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:05,566 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:05,566 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:05,566 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,566 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,566 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,566 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,566 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,689 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,689 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,689 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,689 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,689 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,824 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,824 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,824 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,824 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,824 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,947 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,947 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,947 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,947 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:05,947 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,067 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,067 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,067 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,067 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,067 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,193 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,193 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,193 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,193 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,193 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,197 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:06,197 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:06,197 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:06,197 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:06,197 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:06,197 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,197 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,197 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,197 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,197 - INFO: check station DEBW107 (1 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,327 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,327 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,327 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,327 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,327 - INFO: check station DEBY081 (2 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,455 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,455 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,455 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,455 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,455 - INFO: check station DEBW013 (3 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,596 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,596 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,596 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,596 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,596 - INFO: check station DEBW076 (4 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,741 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,741 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,741 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,741 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,741 - INFO: check station DEBW087 (5 / 5) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:06,897 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,897 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,897 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,897 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:06,897 - INFO: run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
INFO root:pre_processing.py:189 check valid stations started (all) INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:203 DEBW107: history_shape = (3955, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:203 DEBY081: history_shape = (5386, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:203 DEBW013: history_shape = (3954, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:203 DEBW076: history_shape = (5410, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:203 DEBW087: history_shape = (5828, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. DEBUG root:pre_processing.py:165 Train stations (len=5): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087'] INFO root:pre_processing.py:189 check valid stations started (train) DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (2128, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (3948, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (2144, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (3967, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (4001, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:207 run for 0:00:02 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:165 Val stations (len=5): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087'] INFO root:pre_processing.py:189 check valid stations started (val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (692, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (698, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. DEBUG root:pre_processing.py:165 Test stations (len=5): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087'] INFO root:pre_processing.py:189 check valid stations started (test) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (714, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (696, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. DEBUG root:pre_processing.py:165 Train_val stations (len=5): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087'] INFO root:pre_processing.py:189 check valid stations started (train_val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (2859, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (4656, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (2858, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (4698, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 5) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (4732, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 5 station(s). Found 5/5 valid stations. DEBUG root:pre_processing.py:50 ######################################## DEBUG root:pre_processing.py:55 Number of all stations: 15 DEBUG root:pre_processing.py:56 Number of training stations: 5 DEBUG root:pre_processing.py:57 Number of val stations: 5 DEBUG root:pre_processing.py:58 Number of test stations: 5 DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:59 TEST SHAPE OF GENERATOR CALL: (1080, 14, 1, 2)(1080, 3) DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/latex_report DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:data_generator.py:228 load pickle data from /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2010-01-01_2017-12-31_.pickle
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_split_train_val_test 5.58
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:07,065 - INFO: PreProcessing finished after 0:00:05 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,065 - INFO: PreProcessing finished after 0:00:05 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,065 - INFO: PreProcessing finished after 0:00:05 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,065 - INFO: PreProcessing finished after 0:00:05 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,065 - INFO: PreProcessing finished after 0:00:05 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,067 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:07,067 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:07,067 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:07,067 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:07,067 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:07,075 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,075 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,075 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,075 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,075 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:07,075 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:07,075 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:07,075 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:07,075 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:07,075 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:07,079 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:07,079 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:07,079 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:07,079 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:07,079 - INFO: check valid stations started (train) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:07,287 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:07,287 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:07,287 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:07,287 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:07,287 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:07,379 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,379 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,379 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,379 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,379 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,550 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,550 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,550 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,550 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,550 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,715 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,715 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,715 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,715 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,715 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,840 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,840 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,840 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,840 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,840 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,969 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,969 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,969 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,969 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:07,969 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,094 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,094 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,094 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,094 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,094 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,095 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:08,095 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:08,095 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:08,095 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:08,095 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:08,170 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:08,170 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:08,170 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:08,170 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:08,170 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:08,426 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:08,426 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:08,426 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:08,426 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:08,426 - INFO: check valid stations started (val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:08,426 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,426 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,426 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,426 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,426 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,559 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,559 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,559 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,559 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,559 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,676 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,676 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,676 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,676 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,676 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,799 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,799 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,799 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,799 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,799 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,919 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,919 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,919 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,919 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:08,919 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,044 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,044 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,044 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,044 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,044 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,045 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,045 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,045 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,045 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,045 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,125 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:09,125 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:09,125 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:09,125 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:09,125 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:09,134 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:09,134 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:09,134 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:09,134 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:09,134 - INFO: check valid stations started (test) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:09,135 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,135 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,135 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,135 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,135 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,292 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,292 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,292 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,292 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,292 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,432 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,432 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,432 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,432 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,432 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,575 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,575 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,575 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,575 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,575 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,713 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,713 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,713 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,713 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,713 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,847 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,847 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,847 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,847 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,847 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:09,848 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,848 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,848 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,848 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:09,848 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:11,764 - INFO: run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:11,764 - INFO: run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:11,764 - INFO: run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:11,764 - INFO: run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:11,764 - INFO: run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:11,778 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:11,778 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:11,778 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:11,778 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:11,778 - INFO: check valid stations started (train_val) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:11,778 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,778 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,778 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,778 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,778 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,957 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,957 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,957 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,957 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:11,957 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,118 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,118 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,118 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,118 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,118 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,261 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,261 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,261 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,261 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,261 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,408 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,408 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,408 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,408 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,408 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,562 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,562 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,562 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,562 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,562 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,563 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,563 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,563 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,563 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,563 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,651 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:12,651 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:12,651 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:12,651 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:12,651 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
DEBUG root:pre_processing.py:165 Train stations (len=6): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] INFO root:pre_processing.py:189 check valid stations started (train) DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (2128, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (3948, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (2144, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (3967, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2007-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (4001, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:pre_processing.py:165 Val stations (len=6): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] INFO root:pre_processing.py:189 check valid stations started (val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (692, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (698, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2008-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (715, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. DEBUG root:pre_processing.py:165 Test stations (len=6): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] INFO root:pre_processing.py:189 check valid stations started (test) INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (714, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (696, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_2010-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (1080, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:03 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. DEBUG root:pre_processing.py:165 Train_val stations (len=6): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] INFO root:pre_processing.py:189 check valid stations started (train_val) INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (2859, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (4656, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (2858, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (4698, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2009-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (4732, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations.
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_create_set_split_not_all_stations 0.70
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:12,663 - INFO: PreProcessing finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,663 - INFO: PreProcessing finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,663 - INFO: PreProcessing finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,663 - INFO: PreProcessing finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,663 - INFO: PreProcessing finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,665 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:12,665 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:12,665 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:12,665 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:12,665 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:12,672 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,672 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,672 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,672 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,672 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:12,673 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:12,673 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:12,673 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:12,673 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:12,673 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:12,676 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:12,676 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:12,676 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:12,676 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:12,676 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:12,894 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,894 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,894 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,894 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,894 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:12,974 - INFO: check station DEBW107 (1 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,974 - INFO: check station DEBW107 (1 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,974 - INFO: check station DEBW107 (1 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,974 - INFO: check station DEBW107 (1 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:12,974 - INFO: check station DEBW107 (1 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,129 - INFO: check station DEBY081 (2 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,129 - INFO: check station DEBY081 (2 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,129 - INFO: check station DEBY081 (2 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,129 - INFO: check station DEBY081 (2 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,129 - INFO: check station DEBY081 (2 / 2) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,270 - INFO: run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:13,270 - INFO: run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:13,270 - INFO: run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:13,270 - INFO: run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:13,270 - INFO: run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
DEBUG root:pre_processing.py:165 Awesome stations (len=2): ['DEBW107', 'DEBY081'] INFO root:pre_processing.py:189 check valid stations started (awesome) DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:196 check station DEBW107 (1 / 2) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (3955, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 2) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (5386, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 2 station(s). Found 2/2 valid stations. DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_create_set_split_all_stations 1.36
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:13,376 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,376 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,376 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,376 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,376 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,378 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:13,378 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:13,378 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:13,378 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:13,378 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:13,387 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,387 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,387 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,387 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,387 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:13,387 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:13,387 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:13,387 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:13,387 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:13,387 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:13,391 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:13,391 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:13,391 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:13,391 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:13,391 - INFO: check valid stations started (awesome) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:13,610 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:13,610 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:13,610 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:13,610 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:13,610 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:13,695 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,695 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,695 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,695 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,695 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,838 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,838 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,838 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,838 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,838 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,987 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,987 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,987 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,987 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:13,987 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,144 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,144 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,144 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,144 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,144 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,296 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,296 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,296 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,296 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,296 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,447 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,447 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,447 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,447 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,447 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:14,448 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,448 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,448 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,448 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,448 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,531 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:14,531 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:14,531 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:14,531 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:14,531 - INFO: run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
DEBUG root:pre_processing.py:165 Awesome stations (len=6): ['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] INFO root:pre_processing.py:189 check valid stations started (awesome) DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW107_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW107: history_shape = (3955, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW107: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBY081 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBY081_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBY081: history_shape = (5386, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBY081: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW013 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW013_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW013: history_shape = (3954, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW013: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW076 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW076_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW076: history_shape = (5410, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW076: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW087 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:data_generator.py:213 save pickle data to /home/root/machinelearningtools/data/toar_daily/tmp/DEBW087_o3_temp_1997-01-01_2017-12-31_.pickle DEBUG root:pre_processing.py:203 DEBW087: history_shape = (5828, 14, 1, 2) DEBUG root:pre_processing.py:204 DEBW087: loading time = 0:00:01 (hh:mm:ss) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) DEBUG root:data_generator.py:185 load not pickle data for DEBW001 DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW001&station_type=background&network_name=AIRBASE HTTP/1.1" 200 None INFO root:pre_processing.py:207 run for 0:00:02 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW107_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBY081_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW013_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW076_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily DEBUG root:data_preparation.py:94 try to load local data from: /home/root/machinelearningtools/data/toar_daily/DEBW087_o3_temp.nc DEBUG root:data_preparation.py:100 loading finished
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_check_valid_stations[None] 0.40
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:14,756 - INFO: PreProcessing finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,756 - INFO: PreProcessing finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,756 - INFO: PreProcessing finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,756 - INFO: PreProcessing finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,756 - INFO: PreProcessing finished after 0:00:02 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,757 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:14,757 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:14,757 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:14,757 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:14,757 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:14,766 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,766 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,766 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,766 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,766 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:14,766 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:14,766 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:14,766 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:14,766 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:14,766 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:14,769 - INFO: check valid stations started [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:14,769 - INFO: check valid stations started [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:14,769 - INFO: check valid stations started [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:14,769 - INFO: check valid stations started [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:14,769 - INFO: check valid stations started [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:14,983 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,983 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,983 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,983 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:14,983 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,060 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,060 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,060 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,060 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,060 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,068 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,068 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,068 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,068 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,068 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,074 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,074 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,074 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,074 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,074 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,079 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,079 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,079 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,079 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,079 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,085 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,085 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,085 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,085 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,085 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,090 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,090 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,090 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,090 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,090 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,092 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,092 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,092 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,092 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,092 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,166 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,166 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,166 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,166 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,166 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
INFO root:pre_processing.py:189 check valid stations started WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations.
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_check_valid_stations[tester] 0.42
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:15,178 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,178 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,178 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,178 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,178 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,181 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:15,181 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:15,181 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:15,181 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:15,181 - INFO: ExperimentSetup started [run_environment.py:__init__:30] 2020-10-27 14:09:15,192 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,192 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,192 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,192 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,192 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,192 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,192 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,192 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,192 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,192 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 ExperimentSetup started DEBUG root:helpers.py:149 set path to: /home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: data_path(general)=/home/root/machinelearningtools/data/toar_daily/ DEBUG root:experiment_setup.py:155 set experiment attribute: hostname(general)=runner-RZobXyF8-project-2411-concurrent-0 DEBUG root:experiment_setup.py:155 set experiment attribute: hpc_hosts(general)=['jw', 'hdfmlc', 'ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: login_nodes(general)=['ju', 'hdfmll'] DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_model(general)=True DEBUG root:helpers.py:35 Path already exists: /home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: bootstrap_path(general)=/home/root/machinelearningtools/data/toar_daily/../bootstrap_daily DEBUG root:experiment_setup.py:155 set experiment attribute: trainable(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: fraction_of_training(general)=0.8 DEBUG root:experiment_setup.py:155 set experiment attribute: extreme_values(train)=None DEBUG root:experiment_setup.py:155 set experiment attribute: extremes_on_right_tail_only(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: upsampling(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: permute_data(train)=False DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_name(general)=TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: experiment_path(general)=/builds/toar/mlair/TestExperiment DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment DEBUG root:experiment_setup.py:155 set experiment attribute: plot_path(general)=/builds/toar/mlair/TestExperiment/plots DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/plots DEBUG root:experiment_setup.py:155 set experiment attribute: forecast_path(general)=/builds/toar/mlair/TestExperiment/forecasts DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/TestExperiment/forecasts DEBUG root:experiment_setup.py:155 set experiment attribute: stations(general)=['DEBW107', 'DEBY081', 'DEBW013', 'DEBW076', 'DEBW087', 'DEBW001'] DEBUG root:experiment_setup.py:155 set experiment attribute: network(general)=AIRBASE DEBUG root:experiment_setup.py:155 set experiment attribute: station_type(general)=background DEBUG root:experiment_setup.py:155 set experiment attribute: statistics_per_var(general)={'o3': 'dma8eu', 'temp': 'maximum'} DEBUG root:experiment_setup.py:155 set experiment attribute: variables(general)=['o3', 'temp'] DEBUG root:experiment_setup.py:172 check if all variables are included in statistics_per_var DEBUG root:experiment_setup.py:155 set experiment attribute: start(general)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(general)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: window_history_size(general)=13 DEBUG root:experiment_setup.py:155 set experiment attribute: overwrite_local_data(preprocessing)=False DEBUG root:experiment_setup.py:155 set experiment attribute: sampling(general)=daily DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(general)={'scope': 'data', 'method': 'standardise', 'mean': 'estimate'} DEBUG root:experiment_setup.py:155 set experiment attribute: transformation(preprocessing)=None DEBUG root:experiment_setup.py:155 set experiment attribute: target_var(general)=o3 DEBUG root:experiment_setup.py:155 set experiment attribute: target_dim(general)=variables DEBUG root:experiment_setup.py:155 set experiment attribute: window_lead_time(general)=3 DEBUG root:experiment_setup.py:155 set experiment attribute: dimensions(general)={'new_index': ['datetime', 'Stations']} DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_dim(general)=datetime DEBUG root:experiment_setup.py:155 set experiment attribute: interpolate_method(general)=linear DEBUG root:experiment_setup.py:155 set experiment attribute: limit_nan_fill(general)=1 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train)=2007-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(val)=2008-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(val)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(test)=2010-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(test)=2017-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(test)=90 DEBUG root:experiment_setup.py:155 set experiment attribute: start(train_val)=1997-01-01 DEBUG root:experiment_setup.py:155 set experiment attribute: end(train_val)=2009-12-31 DEBUG root:experiment_setup.py:155 set experiment attribute: min_length(train_val)=180 DEBUG root:experiment_setup.py:155 set experiment attribute: use_all_stations_on_all_data_sets(general)=True DEBUG root:experiment_setup.py:155 set experiment attribute: evaluate_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: create_new_bootstraps(general.postprocessing)=True DEBUG root:experiment_setup.py:155 set experiment attribute: number_of_bootstraps(general.postprocessing)=20 DEBUG root:experiment_setup.py:155 set experiment attribute: plot_list(general.postprocessing)=['PlotMonthlySummary', 'PlotStationMap', 'PlotClimatologicalSkillScore', 'PlotTimeSeries', 'PlotCompetitiveSkillScore', 'PlotBootstrapSkillScore', 'PlotConditionalQuantiles', 'PlotAvailability', 'PlotAvailabilityHistogram'] INFO root:run_environment.py:39 ExperimentSetup finished after 0:00:01 (hh:mm:ss) INFO root:run_environment.py:30 PreProcessing started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:15,195 - INFO: check valid stations started (tester) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:15,195 - INFO: check valid stations started (tester) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:15,195 - INFO: check valid stations started (tester) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:15,195 - INFO: check valid stations started (tester) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:15,195 - INFO: check valid stations started (tester) [pre_processing.py:check_valid_stations:189] 2020-10-27 14:09:15,427 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,427 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,427 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,427 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,427 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,503 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,503 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,503 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,503 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,503 - INFO: check station DEBW107 (1 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,512 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,512 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,512 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,512 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,512 - INFO: check station DEBY081 (2 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,518 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,518 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,518 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,518 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,518 - INFO: check station DEBW013 (3 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,523 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,523 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,523 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,523 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,523 - INFO: check station DEBW076 (4 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,528 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,528 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,528 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,528 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,528 - INFO: check station DEBW087 (5 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,534 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,534 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,534 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,534 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,534 - INFO: check station DEBW001 (6 / 6) [pre_processing.py:check_valid_stations:196] 2020-10-27 14:09:15,535 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,535 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,535 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,535 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,535 - WARNING: [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:15,615 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,615 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,615 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,615 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207] 2020-10-27 14:09:15,615 - INFO: run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations. [pre_processing.py:check_valid_stations:207]
-------------------------------Captured log call--------------------------------
INFO root:pre_processing.py:189 check valid stations started (tester) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' INFO root:pre_processing.py:196 check station DEBW107 (1 / 6) INFO root:pre_processing.py:196 check station DEBY081 (2 / 6) INFO root:pre_processing.py:196 check station DEBW013 (3 / 6) INFO root:pre_processing.py:196 check station DEBW076 (4 / 6) INFO root:pre_processing.py:196 check station DEBW087 (5 / 6) INFO root:pre_processing.py:196 check station DEBW001 (6 / 6) WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/home/root/machinelearningtools/data/toar_daily/DEBW001_o3_temp.nc' INFO root:pre_processing.py:207 run for 0:00:01 (hh:mm:ss) to check 6 station(s). Found 5/6 valid stations.
Passed test/test_modules/test_pre_processing.py::TestPreProcessing::test_split_set_indices 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:15,622 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,622 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,622 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,622 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,622 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,624 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,624 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,624 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,624 - INFO: PreProcessing started [run_environment.py:__init__:30] 2020-10-27 14:09:15,624 - INFO: PreProcessing started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
INFO root:run_environment.py:30 PreProcessing started
Passed test/test_modules/test_run_environment.py::TestRunEnvironment::test_enter 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:15,631 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,631 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,631 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,631 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,631 - INFO: PreProcessing finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:15,633 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,633 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,633 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,633 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,633 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,633 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,633 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,633 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,633 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,633 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_run_environment.py::TestRunEnvironment::test_exit 0.10
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:15,636 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,636 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,636 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,636 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,636 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,736 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,736 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,736 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,736 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,736 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_run_environment.py::TestRunEnvironment::test_init 0.00
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:15,742 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,742 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,742 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,742 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,742 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,742 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,742 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,742 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,742 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,742 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_run_environment.py::TestRunEnvironment::test_del 0.20
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:15,747 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,747 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,747 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,747 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,747 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:15,947 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,947 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,947 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,947 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:15,947 - INFO: RunEnvironment finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 RunEnvironment started INFO root:run_environment.py:39 RunEnvironment finished after 0:00:01 (hh:mm:ss)
Passed test/test_modules/test_training.py::TestTraining::test_init 13.34
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:16,284 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:16,284 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:16,284 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:16,284 - INFO: RunEnvironment started [run_environment.py:__init__:30] 2020-10-27 14:09:16,284 - INFO: RunEnvironment started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 RunEnvironment started
------------------------------Captured stdout call------------------------------
Epoch 1/2 - 4s - loss: 27.4873 - val_loss: 10.2356 Epoch 2/2 - 0s - loss: 17.6485 - val_loss: 10.4213
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:16,316 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:16,316 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:16,316 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:16,316 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:16,316 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:16,321 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:16,321 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:16,321 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:16,321 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:16,321 - WARNING: [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' [data_preparation.py:load_data:102] 2020-10-27 14:09:16,396 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:09:16,396 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:09:16,396 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:09:16,396 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:09:16,396 - INFO: load: o3 [join.py:download_join:59] 2020-10-27 14:09:17,826 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:09:17,826 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:09:17,826 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:09:17,826 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:09:17,826 - INFO: load: temp [join.py:download_join:59] 2020-10-27 14:09:20,413 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:20,413 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:20,413 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:20,413 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:20,413 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:20,413 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:20,413 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:20,413 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:20,413 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:20,413 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:20,413 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:20,413 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:20,413 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:20,413 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:20,413 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:23,854 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:23,854 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:23,854 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:23,854 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:23,854 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:28,263 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:28,263 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:28,263 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:28,263 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:28,263 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:29,184 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:29,184 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:29,184 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:29,184 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:29,184 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:29,659 - INFO: Training finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,659 - INFO: Training finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,659 - INFO: Training finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,659 - INFO: Training finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,659 - INFO: Training finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39]
-------------------------------Captured log call--------------------------------
INFO root:run_environment.py:30 Training started DEBUG root:data_generator.py:185 load not pickle data for DEBW107 DEBUG root:helpers.py:35 Path already exists: /builds/toar/mlair/test/test_modules/data DEBUG root:data_preparation.py:94 try to load local data from: /builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc WARNING root:data_preparation.py:102 [Errno 2] No such file or directory: '/builds/toar/mlair/test/test_modules/data/DEBW107_o3_temp.nc' DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/series/?station_id=DEBW107&network_name=AIRBASE HTTP/1.1" 200 None INFO root:join.py:59 load: o3 DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=23031&statistics=dma8eu&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: o3 INFO root:join.py:59 load: temp DEBUG urllib3.connectionpool:connectionpool.py:959 Starting new HTTPS connection (1): join.fz-juelich.de:443 DEBUG urllib3.connectionpool:connectionpool.py:437 https://join.fz-juelich.de:443 "GET /services/rest/surfacedata/stats/?id=85584&statistics=maximum&sampling=daily&capture=0&min_data_length=1460&format=json HTTP/1.1" 200 None DEBUG root:join.py:80 finished: temp DEBUG root:data_preparation.py:104 loaded new data from JOIN DEBUG root:data_generator.py:213 save pickle data to /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:training.py:91 Train with 16 mini batches. INFO root:training.py:92 Train with option upsampling=False. INFO root:training.py:93 Train with option data_permutation=False. DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:training.py:156 saving callbacks DEBUG root:training.py:142 load best model: /builds/toar/mlair/test/test_modules/TestExperiment/model_checkpoint INFO root:training.py:145 reload weights... DEBUG matplotlib.font_manager:font_manager.py:1246 findfont: Matching sans\-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0. DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal 700 normal>) = 0.33499999999999996 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 1.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal 700 normal>) = 1.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 0.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1281 findfont: Matching sans\-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to DejaVu Sans ('/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf') with score of 0.050000. DEBUG matplotlib.font_manager:font_manager.py:1246 findfont: Matching sans\-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0. DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal 700 normal>) = 0.33499999999999996 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 1.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal 700 normal>) = 1.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 0.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal 700 normal>) = 10.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal 700 normal>) = 11.335 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1258 findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05 DEBUG matplotlib.font_manager:font_manager.py:1281 findfont: Matching sans\-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0 to DejaVu Sans ('/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf') with score of 0.050000. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG root:training.py:133 save best model to /builds/toar/mlair/test/test_modules/TestExperiment/test_model.h5 INFO root:run_environment.py:39 Training finished after 0:00:14 (hh:mm:ss)
Passed test/test_modules/test_training.py::TestTraining::test_run 9.10
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:29,664 - INFO: RunEnvironment finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,664 - INFO: RunEnvironment finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,664 - INFO: RunEnvironment finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,664 - INFO: RunEnvironment finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,664 - INFO: RunEnvironment finished after 0:00:14 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:29,664 - INFO: Copy log file to /builds/toar/mlair/test/test_modules/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:29,664 - INFO: Copy log file to /builds/toar/mlair/test/test_modules/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:29,664 - INFO: Copy log file to /builds/toar/mlair/test/test_modules/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:29,664 - INFO: Copy log file to /builds/toar/mlair/test/test_modules/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:29,664 - INFO: Copy log file to /builds/toar/mlair/test/test_modules/TestExperiment/logging_000.log [run_environment.py:__copy_log_file:61] 2020-10-27 14:09:29,940 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:29,940 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:29,940 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:29,940 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:29,940 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
------------------------------Captured stdout call------------------------------
Epoch 1/2 - 3s - loss: 21.0448 - val_loss: 32.3770 Epoch 2/2 - 0s - loss: 20.7716 - val_loss: 11.3701
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:29,985 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:29,985 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:29,985 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:29,985 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:29,985 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:29,985 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:29,985 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:29,985 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:29,985 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:29,985 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:29,986 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:29,986 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:29,986 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:29,986 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:29,986 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:32,645 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:32,645 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:32,645 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:32,645 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:32,645 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:37,452 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:37,452 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:37,452 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:37,452 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:37,452 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:38,611 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:38,611 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:38,611 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:38,611 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:38,611 - INFO: reload weights... [training.py:load_best_model:145]
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:training.py:91 Train with 16 mini batches. INFO root:training.py:92 Train with option upsampling=False. INFO root:training.py:93 Train with option data_permutation=False. DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:training.py:156 saving callbacks DEBUG root:training.py:142 load best model: /builds/toar/mlair/test/test_modules/TestExperiment/model_checkpoint INFO root:training.py:145 reload weights... DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG root:training.py:133 save best model to /builds/toar/mlair/test/test_modules/TestExperiment/test_model.h5
Passed test/test_modules/test_training.py::TestTraining::test_make_predict_function 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:39,073 - INFO: Training finished after 0:00:10 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,073 - INFO: Training finished after 0:00:10 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,073 - INFO: Training finished after 0:00:10 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,073 - INFO: Training finished after 0:00:10 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,073 - INFO: Training finished after 0:00:10 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,415 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,415 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,415 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,415 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,415 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
Passed test/test_modules/test_training.py::TestTraining::test_set_gen 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:39,422 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,422 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,422 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,422 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,422 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,735 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,735 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,735 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,735 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:39,735 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
Passed test/test_modules/test_training.py::TestTraining::test_set_generators 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:39,740 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,740 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,740 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,740 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:39,740 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,050 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,050 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,050 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,050 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,050 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
Passed test/test_modules/test_training.py::TestTraining::test_train 10.12
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:40,056 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,056 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,056 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,056 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,056 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:40,317 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,317 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,317 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,317 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:40,317 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
------------------------------Captured stdout call------------------------------
Epoch 1/2 - 3s - loss: 23.6781 - val_loss: 32.0804 Epoch 2/2 - 0s - loss: 21.3786 - val_loss: 17.3183
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:40,363 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:40,363 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:40,363 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:40,363 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:40,363 - INFO: Train with 16 mini batches. [training.py:train:91] 2020-10-27 14:09:40,363 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:40,363 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:40,363 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:40,363 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:40,363 - INFO: Train with option upsampling=False. [training.py:train:92] 2020-10-27 14:09:40,363 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:40,363 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:40,363 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:40,363 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:40,363 - INFO: Train with option data_permutation=False. [training.py:train:93] 2020-10-27 14:09:43,226 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:43,226 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:43,226 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:43,226 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:43,226 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:48,867 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:48,867 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:48,867 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:48,867 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:48,867 - INFO: Set learning rate to 0.01 [keras_extensions.py:on_epoch_begin:90] 2020-10-27 14:09:50,138 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:50,138 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:50,138 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:50,138 - INFO: reload weights... [training.py:load_best_model:145] 2020-10-27 14:09:50,138 - INFO: reload weights... [training.py:load_best_model:145]
-------------------------------Captured log call--------------------------------
DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:training.py:91 Train with 16 mini batches. INFO root:training.py:92 Train with option upsampling=False. INFO root:training.py:93 Train with option data_permutation=False. DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle INFO root:keras_extensions.py:90 Set learning rate to 0.01 DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:data_generator.py:228 load pickle data from /builds/toar/mlair/test/test_modules/data/tmp/DEBW107_o3_temp_None_None_.pickle DEBUG root:training.py:156 saving callbacks DEBUG root:training.py:142 load best model: /builds/toar/mlair/test/test_modules/TestExperiment/model_checkpoint INFO root:training.py:145 reload weights... DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_modules/test_training.py::TestTraining::test_save_model 5.39
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:50,477 - INFO: Training finished after 0:00:11 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:50,477 - INFO: Training finished after 0:00:11 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:50,477 - INFO: Training finished after 0:00:11 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:50,477 - INFO: Training finished after 0:00:11 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:50,477 - INFO: Training finished after 0:00:11 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:50,768 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:50,768 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:50,768 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:50,768 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:50,768 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:133 save best model to /builds/toar/mlair/test/test_modules/TestExperiment/test_model.h5
Passed test/test_modules/test_training.py::TestTraining::test_load_best_model_no_weights 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:56,167 - INFO: Training finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,167 - INFO: Training finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,167 - INFO: Training finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,167 - INFO: Training finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,167 - INFO: Training finished after 0:00:06 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,483 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,483 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,483 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,483 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,483 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
------------------------------Captured stderr call------------------------------
2020-10-27 14:09:56,486 - INFO: no weights to reload... [training.py:load_best_model:147] 2020-10-27 14:09:56,486 - INFO: no weights to reload... [training.py:load_best_model:147] 2020-10-27 14:09:56,486 - INFO: no weights to reload... [training.py:load_best_model:147] 2020-10-27 14:09:56,486 - INFO: no weights to reload... [training.py:load_best_model:147] 2020-10-27 14:09:56,486 - INFO: no weights to reload... [training.py:load_best_model:147]
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:142 load best model: notExisting INFO root:training.py:147 no weights to reload...
Passed test/test_modules/test_training.py::TestTraining::test_save_callbacks_history_created 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:56,489 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,489 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,489 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,489 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,489 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,772 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,772 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,772 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,772 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:56,772 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:156 saving callbacks
Passed test/test_modules/test_training.py::TestTraining::test_save_callbacks_lr_created 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:56,779 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,779 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,779 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,779 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:56,779 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,581 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,581 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,581 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,581 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,581 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:156 saving callbacks
Passed test/test_modules/test_training.py::TestTraining::test_save_callbacks_inspect_history 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:57,587 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,587 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,587 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,587 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,587 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,935 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,935 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,935 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,935 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:57,935 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:156 saving callbacks
Passed test/test_modules/test_training.py::TestTraining::test_save_callbacks_inspect_lr 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:57,942 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,942 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,942 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,942 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:57,942 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,261 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,261 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,261 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,261 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,261 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG root:training.py:156 saving callbacks
Passed test/test_modules/test_training.py::TestTraining::test_create_monitoring_plots 0.36
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:58,268 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,268 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,268 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,268 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,268 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,591 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,591 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,591 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,591 - INFO: Training started [run_environment.py:__init__:30] 2020-10-27 14:09:58,591 - INFO: Training started [run_environment.py:__init__:30]
-------------------------------Captured log setup-------------------------------
DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> DEBUG root:inception_model.py:58 Inception Block with activation: <class 'keras.layers.advanced_activations.PReLU'> INFO root:run_environment.py:30 Training started
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_get_plot_metric 0.00
-----------------------------Captured stderr setup------------------------------
2020-10-27 14:09:58,952 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,952 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,952 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,952 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39] 2020-10-27 14:09:58,952 - INFO: Training finished after 0:00:01 (hh:mm:ss) [run_environment.py:__del__:39]
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_get_plot_metric_short_metric 0.00
No log output captured.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_get_plot_metric_main_branch 0.00
No log output captured.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_filter_columns 0.00
No log output captured.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_plot_from_hist_obj 0.16
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_plot_from_hist_dict 0.18
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelHistory::test_plot_additional_loss 0.28
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelLearningRate::test_plot_from_lr_obj 0.16
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.
Passed test/test_plotting/test_training_monitoring.py::TestPlotModelLearningRate::test_plot_from_lr_dict 0.18
-------------------------------Captured log call--------------------------------
DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:677 Assigning font /b'F1' = '/usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf' DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:719 Embedding font /usr/lib64/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf. DEBUG matplotlib.backends.backend_pdf:backend_pdf.py:726 Writing TrueType font.