{ "cells": [ { "cell_type": "markdown", "id": "de3399ff", "metadata": {}, "source": [ "# Explanation Scores: Falkonry API helper notebook\n", "This notebook connects to Falkonry REST API, resolves datastream/assessment/model, lists events, and extracts model `signalInfos` to build a list of signals that should be kept in the model and which ones should be removed based on learnings behavior.\n", "\n", "### 9 Steps to go live with your custom Calculations\n", "1. Set Up Falkonry API Access\n", "2. Helpers: authenticated GET and name lookup\n", "3. Resolve Datastream, Assessment, and Model IDs \n", "4. List events (learn ranges) for the assessment and summarize start/end times\n", "5. Page through Explanation output signals (10 at a time) and collect signal id + name\n", "6. (OPTIONAL) Page through Prediction output signals (10 at a time) and collect signal id + name\n", "7. (OPTIONAL) Save outputs to CSV files (if available)\n", "8. Fetch per-event per-signal explanation cells\n", "9. Transpose to desired format and reduce the precision to 3 decimals\n", "\n", "#### Example output\n", "\n", "| draftback_signals | median_exp_score | use in model |\n", "|:------------------|:----------------:|:-------------|\n", "| LIT_759_1\t| 1.0 | keep |\n", "| LIT_759_2 | 1.0 | keep |\n", "| LIT_759_3 | 1.0 | keep |\n", "| BlastPressureHot | 1.0 | keep |\n", "| FY_401_3 | 1.0 | keep |\n", "| CALC_Differential_Pressure | 1.0 | keep |\n", "| CALC_Charges_Per_Hour | 1.0 | keep |\n", "| FT_Gas_Analyzer2_CO2 | 1.0 | keep |\n", "| FT_Gas_Analyzer2_CO | 1.0 | keep |\n", "| SOMA_ZONE_AVG | 0.75 | keep |\n", "| TE_412_A | 0.32 | keep |\n", "| TE_412_C | 0.036 | remove |\n", "| TE_412_B | 0.035 | remove |\n", "| CALC_EquivWindTHM | 0.003 | remove |\n", "| FT_Gas_Analyzer2_EFF | -0.327 | remove |" ] }, { "cell_type": "markdown", "id": "1f79010a", "metadata": {}, "source": [ "COPYRIGHT (c) 2026 FALKONRY, INC. ALL RIGHTS RESERVED.\n", "\n", "FOR FALKONRY'S CUSTOMER USE ONLY.\n", "\n", "THIS SAMPLE CODE FROM FALKONRY IS PROVIDED \"AS IS\" AND WARRANTIES OF ANY KIND, \n", "EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF \n", "MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO \n", "EVENT SHALL FALKONRY INC. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, \n", "SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, \n", "LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) SUSTAINED BY YOU OR\n", "A THIRD PARTY, HOWEVER CAUSED AND UNDER ANY THEORY OF LIABILITY, WHETHER IN \n", "CONTRACT, STRICT LIABILITY, OR TORT ARISING IN ANY WAY OUT OF THE USE OF OR \n", "INABILITY TO USE THIS SAMPLE CODE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH \n", "DAMAGE." ] }, { "cell_type": "markdown", "id": "121b4164", "metadata": {}, "source": [ "## 1. Set Up Falkonry API Access\n", "\n", "Configure authentication and base URL for Falkonry API requests. Set your API key, account_id, and other required variables." ] }, { "cell_type": "code", "execution_count": null, "id": "10f110a5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Testing API connection...\n", "https://stelco-api.falkonry.ai/api/1.2/accounts/1085360747420786688\n", "Status code: 200\n", "API connection successful!\n", "Proceed with the rest of the notebook...\n" ] } ], "source": [ "import requests\n", "import pandas as pd\n", "from urllib.parse import urljoin\n", "import getpass\n", "from typing import Optional, Dict, Any\n", "\n", "# --- Edit these (or leave blank to be prompted) ---\n", "BASE_SERVER = '' # e.g. app3.falkonry.ai'\n", "BASE_URL = f'https://{BASE_SERVER}/api/1.2' # e.g. https://app3.falkonry.ai/api/1.2\n", "ACCOUNT_ID = '' # set your account id (tenant)\n", "API_TOKEN = '' # leave blank to prompt securely\n", "MODEL_NAME = '' # e.g. M[2] set the name of the model to analyze\n", "ASSESSMENT_NAME = ''\n", "DATASTREAM_NAME = ''\n", "ENTITY_NAME = 'BF'\n", "EVENTS_OF_INTEREST = [] # e.g. ['channeling', 'draftback', 'instability'] # specify event types to include in analysis (e.g. LearnRange, PredictRange, etc.)\n", "EXTRACT_LEARN_EVENTS = True\n", "SIGNAL_PREFIX = \"\" # prefix to remove from signal names in final output (adjust as needed)\n", "EXPLANATION_THRESHOLD = 0.2\n", "LEVEL = 12 # 11 = 1-minute, 12 = 10-minute, 13 = hourly, 14 = daily\n", "HEADERS = {\n", " \"Authorization\": f\"Bearer {API_TOKEN}\",\n", " \"Content-Type\": \"application/json\"\n", "}\n", "TIMEOUT = 30\n", "\n", "# Test the API connection before proceeding with the rest of the notebook\n", "print(\"Testing API connection...\")\n", "try:\n", " test_url = f\"{BASE_URL}/accounts/{ACCOUNT_ID}\"\n", " print (test_url)\n", " response = requests.get(test_url, headers=HEADERS, timeout=10)\n", " print(f\"Status code: {response.status_code}\")\n", " if response.status_code == 200:\n", " print(\"API connection successful!\")\n", " print(\"Proceed with the rest of the notebook...\")\n", " else:\n", " print(f\"API error: {response.text[:100]}\")\n", " print(\"STOP! Check your API connection parameters and try again.\")\n", "except Exception as e:\n", " print(f\"Connection error: {str(e)}\")\n", " print(\"STOP! Check your API connection parameters and try again.\")" ] }, { "cell_type": "markdown", "id": "8bea1232", "metadata": {}, "source": [ "# 2. Helpers: authenticated GET and name lookup" ] }, { "cell_type": "code", "execution_count": 2, "id": "5161c64e", "metadata": {}, "outputs": [], "source": [ "def api_get(path: str, params: Optional[Dict[str, Any]] = None) -> Any:\n", " url = urljoin(BASE_URL.rstrip('/') + '/', path.lstrip('/'))\n", " resp = requests.get(url, headers=HEADERS, params=params or {}, timeout=TIMEOUT)\n", " resp.raise_for_status()\n", " return resp.json()\n", "\n", "def find_by_name(objs, name, fields=('name','title','id')):\n", " for o in objs or []:\n", " for f in fields:\n", " if f in o and isinstance(o[f], str) and o[f] == name:\n", " return o\n", " return None" ] }, { "cell_type": "markdown", "id": "c697432c", "metadata": {}, "source": [ "# 3. Resolve Datastream, Assessment, and Model IDs" ] }, { "cell_type": "code", "execution_count": 3, "id": "e5df0791", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Found datastream id: 1509650958704091136\n", "Found assessment id: 1511809127735414784\n", "Found model id: 1511817124809330688\n", "Found Entity id: 1509650959211601920\n" ] } ], "source": [ "ds_list = api_get(f'/accounts/{ACCOUNT_ID}/datastreams?limit=1000')\n", "if isinstance(ds_list, dict) and 'items' in ds_list:\n", " ds_items = ds_list['items']\n", "elif isinstance(ds_list, list):\n", " ds_items = ds_list\n", "else:\n", " ds_items = []\n", "datastream = find_by_name(ds_items, DATASTREAM_NAME)\n", "if not datastream:\n", " raise SystemExit(f'Datastream named {DATASTREAM_NAME} not found. Check names and try again.')\n", "datastream_id = datastream.get('id') or datastream.get('_id')\n", "print('Found datastream id:', datastream_id)\n", "\n", "assess_resp = api_get(f'/accounts/{ACCOUNT_ID}/datastreams/{datastream_id}/assessments?limit=1000')\n", "if isinstance(assess_resp, dict) and 'items' in assess_resp:\n", " assess_items = assess_resp['items']\n", "elif isinstance(assess_resp, list):\n", " assess_items = assess_resp\n", "else:\n", " assess_items = []\n", "assessment = find_by_name(assess_items, ASSESSMENT_NAME)\n", "if not assessment:\n", " raise SystemExit(f'Assessment named [{ASSESSMENT_NAME}] not found in datastream [{DATASTREAM_NAME}].')\n", "assessment_id = assessment.get('id') or assessment.get('_id')\n", "print('Found assessment id:', assessment_id)\n", "\n", "models_resp = api_get(f'/accounts/{ACCOUNT_ID}/datastreams/{datastream_id}/assessments/{assessment_id}/models20?limit=1000')\n", "if isinstance(models_resp, dict) and 'items' in models_resp:\n", " model_items = models_resp['items']\n", "elif isinstance(models_resp, list):\n", " model_items = models_resp\n", "else:\n", " model_items = []\n", "model_obj = find_by_name(model_items, MODEL_NAME)\n", "if not model_obj:\n", " print('Warning: model not found by name. Some steps may require model_id to proceed.')\n", " model_id = None\n", "else:\n", " model_id = model_obj.get('id') or model_obj.get('_id')\n", " print('Found model id:', model_id)\n", "\n", "\n", "entity_id = ''\n", "offset = 0\n", "limit = 100\n", "found = []\n", "while True:\n", " params = {'offset': offset, 'limit': limit}\n", " path = f'/accounts/{ACCOUNT_ID}/datastreams/{datastream_id}/entities'\n", " try:\n", " resp = api_get(path, params=params)\n", " except Exception:\n", " break\n", " if isinstance(resp, dict) and 'items' in resp:\n", " items = resp['items']\n", " elif isinstance(resp, list):\n", " items = resp\n", " else:\n", " items = []\n", " if not items:\n", " break\n", " for it in items:\n", " ename = it.get('name') or it.get('title') or ''\n", " if ename.lower() == ENTITY_NAME.lower():\n", " entity_id = it.get('id') or it.get('_id')\n", " print(f'Found Entity id: {entity_id}')\n", " break\n", " offset += limit\n" ] }, { "cell_type": "markdown", "id": "227167b8", "metadata": {}, "source": [ "# 4. List events (learn ranges) for the assessment and summarize start/end times" ] }, { "cell_type": "code", "execution_count": 4, "id": "ba08e14c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Filtered events to labels ['ddraftback', 'unstable', 'channeling']: 11 matches\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "label", "rawType": "object", "type": "string" }, { "name": "start", "rawType": "datetime64[ns, UTC]", "type": "datetime" }, { "name": "end", "rawType": "datetime64[ns, UTC]", "type": "datetime" }, { "name": "event_type", "rawType": "object", "type": "string" } ], "ref": "4dbf3ff5-4e0e-4193-986a-1009cb192063", "rows": [ [ "0", "unstable", "2026-01-19 13:00:00+00:00", "2026-01-19 19:00:00+00:00", "learn" ], [ "7", "channeling", "2026-01-20 20:30:00+00:00", "2026-01-21 00:30:00+00:00", "learn" ], [ "13", "channeling", "2026-01-24 00:30:00+00:00", "2026-01-24 04:30:00+00:00", "learn" ], [ "17", "unstable", "2026-01-29 14:00:00+00:00", "2026-01-29 18:00:00+00:00", "learn" ], [ "18", "channeling", "2026-02-01 22:00:00+00:00", "2026-02-02 01:00:00+00:00", "learn" ], [ "19", "channeling", "2026-02-07 10:00:00+00:00", "2026-02-07 18:00:00+00:00", "learn" ], [ "20", "unstable", "2026-02-23 06:00:00+00:00", "2026-02-23 11:00:00+00:00", "learn" ], [ "22", "unstable", "2026-02-22 11:00:00+00:00", "2026-02-22 18:00:00+00:00", "learn" ], [ "24", "channeling", "2026-02-24 08:00:00+00:00", "2026-02-24 14:00:00+00:00", "learn" ], [ "25", "unstable", "2026-04-22 00:00:00+00:00", "2026-04-22 02:30:00+00:00", "learn" ], [ "34", "unstable", "2026-04-25 09:50:00+00:00", "2026-04-25 11:00:00+00:00", "learn" ] ], "shape": { "columns": 4, "rows": 11 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
labelstartendevent_type
0unstable2026-01-19 13:00:00+00:002026-01-19 19:00:00+00:00learn
7channeling2026-01-20 20:30:00+00:002026-01-21 00:30:00+00:00learn
13channeling2026-01-24 00:30:00+00:002026-01-24 04:30:00+00:00learn
17unstable2026-01-29 14:00:00+00:002026-01-29 18:00:00+00:00learn
18channeling2026-02-01 22:00:00+00:002026-02-02 01:00:00+00:00learn
19channeling2026-02-07 10:00:00+00:002026-02-07 18:00:00+00:00learn
20unstable2026-02-23 06:00:00+00:002026-02-23 11:00:00+00:00learn
22unstable2026-02-22 11:00:00+00:002026-02-22 18:00:00+00:00learn
24channeling2026-02-24 08:00:00+00:002026-02-24 14:00:00+00:00learn
25unstable2026-04-22 00:00:00+00:002026-04-22 02:30:00+00:00learn
34unstable2026-04-25 09:50:00+00:002026-04-25 11:00:00+00:00learn
\n", "
" ], "text/plain": [ " label start end event_type\n", "0 unstable 2026-01-19 13:00:00+00:00 2026-01-19 19:00:00+00:00 learn\n", "7 channeling 2026-01-20 20:30:00+00:00 2026-01-21 00:30:00+00:00 learn\n", "13 channeling 2026-01-24 00:30:00+00:00 2026-01-24 04:30:00+00:00 learn\n", "17 unstable 2026-01-29 14:00:00+00:00 2026-01-29 18:00:00+00:00 learn\n", "18 channeling 2026-02-01 22:00:00+00:00 2026-02-02 01:00:00+00:00 learn\n", "19 channeling 2026-02-07 10:00:00+00:00 2026-02-07 18:00:00+00:00 learn\n", "20 unstable 2026-02-23 06:00:00+00:00 2026-02-23 11:00:00+00:00 learn\n", "22 unstable 2026-02-22 11:00:00+00:00 2026-02-22 18:00:00+00:00 learn\n", "24 channeling 2026-02-24 08:00:00+00:00 2026-02-24 14:00:00+00:00 learn\n", "25 unstable 2026-04-22 00:00:00+00:00 2026-04-22 02:30:00+00:00 learn\n", "34 unstable 2026-04-25 09:50:00+00:00 2026-04-25 11:00:00+00:00 learn" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "if EXTRACT_LEARN_EVENTS:\n", " events_resp = api_get(f'/accounts/{ACCOUNT_ID}/datastreams/{datastream_id}/assessments/{assessment_id}/events?limit=100')\n", " if isinstance(events_resp, dict) and 'items' in events_resp:\n", " events = events_resp['items']\n", " elif isinstance(events_resp, list):\n", " events = events_resp\n", " else:\n", " events = []\n", "\n", " def _summarize_event(e):\n", " label = e.get('value')\n", " start = e.get('time')\n", " end = e.get('end')\n", " return {'label': label, 'start': start, 'end': end, 'event_type': 'learn'}\n", "\n", " events_summary = [_summarize_event(e) for e in events]\n", " df_events = pd.DataFrame(events_summary)\n", " for c in ('start','end'):\n", " try:\n", " df_events[c] = pd.to_datetime(df_events[c], utc=True)\n", " except Exception:\n", " pass\n", " # Filter events by requested labels (case-insensitive).\n", " try:\n", " allowed = set([x.lower() for x in EVENTS_OF_INTEREST]) if 'EVENTS_OF_INTEREST' in globals() and EVENTS_OF_INTEREST else set()\n", " if allowed:\n", " df_events = df_events[df_events['label'].apply(lambda v: isinstance(v, str) and v.lower() in allowed)]\n", " print(f'Filtered events to labels {EVENTS_OF_INTEREST}: {len(df_events)} matches')\n", " else:\n", " print('No EVENTS_OF_INTEREST defined; showing all events')\n", " except Exception as exc:\n", " print('Event label filtering failed:', exc)\n", "else:\n", " df_events = pd.read_csv(\"events-other.csv\")\n", "\n", "df_events" ] }, { "cell_type": "markdown", "id": "7b9fcd7e", "metadata": {}, "source": [ "# 5. Page through Explanation output signals (10 at a time) and collect signal id + name" ] }, { "cell_type": "code", "execution_count": 5, "id": "57aa6573", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collected 15 explanation output signals (paged by 10 )\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "signalId", "rawType": "object", "type": "string" }, { "name": "signalName", "rawType": "object", "type": "string" } ], "ref": "a1c2ea51-f68f-44eb-b90e-08a6ff3db98a", "rows": [ [ "0", "1511824331755118592", "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1" ], [ "1", "1511824331847393281", "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2" ], [ "2", "1511824331872559104", "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3" ], [ "3", "1511824331897724928", "Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Kinetic_Energy" ], [ "4", "1511824331922890752", "Explanation-stelco.ca/lewbfprdpi.BF1_HeatFlux" ], [ "5", "1511824331956445184", "Explanation-stelco.ca/lewbfprdpi.BF1_Stave_Manifold_Heat_Flux_Total" ], [ "6", "1511824331981611008", "Explanation-stelco.ca/lewbfprdpi.BF1_BlastPressureHot" ], [ "7", "1511824332006776832", "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_EFF" ], [ "8", "1511824332027748352", "Explanation-stelco.ca/lewbfprdpi.BF1_FY_401_3" ], [ "9", "1511824332052914176", "Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE_AVG" ], [ "10", "1511824332082274304", "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Differential_Pressure" ], [ "11", "1511824332103245824", "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_EquivWindTHM" ], [ "12", "1511824332128411648", "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Charges_Per_Hour" ], [ "13", "1511824332153577472", "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO2" ], [ "14", "1511824332182937600", "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO" ] ], "shape": { "columns": 2, "rows": 15 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
signalIdsignalName
01511824331755118592Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1
11511824331847393281Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2
21511824331872559104Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3
31511824331897724928Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Ki...
41511824331922890752Explanation-stelco.ca/lewbfprdpi.BF1_HeatFlux
51511824331956445184Explanation-stelco.ca/lewbfprdpi.BF1_Stave_Man...
61511824331981611008Explanation-stelco.ca/lewbfprdpi.BF1_BlastPres...
71511824332006776832Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An...
81511824332027748352Explanation-stelco.ca/lewbfprdpi.BF1_FY_401_3
91511824332052914176Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE...
101511824332082274304Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Diff...
111511824332103245824Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Equi...
121511824332128411648Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Char...
131511824332153577472Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An...
141511824332182937600Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An...
\n", "
" ], "text/plain": [ " signalId signalName\n", "0 1511824331755118592 Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1\n", "1 1511824331847393281 Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2\n", "2 1511824331872559104 Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3\n", "3 1511824331897724928 Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Ki...\n", "4 1511824331922890752 Explanation-stelco.ca/lewbfprdpi.BF1_HeatFlux\n", "5 1511824331956445184 Explanation-stelco.ca/lewbfprdpi.BF1_Stave_Man...\n", "6 1511824331981611008 Explanation-stelco.ca/lewbfprdpi.BF1_BlastPres...\n", "7 1511824332006776832 Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An...\n", "8 1511824332027748352 Explanation-stelco.ca/lewbfprdpi.BF1_FY_401_3\n", "9 1511824332052914176 Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE...\n", "10 1511824332082274304 Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Diff...\n", "11 1511824332103245824 Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Equi...\n", "12 1511824332128411648 Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Char...\n", "13 1511824332153577472 Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An...\n", "14 1511824332182937600 Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_An..." ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "try:\n", " ds_for_signals = datastream_id if 'datastream_id' in globals() and datastream_id else '1485074653623840768'\n", " assess_for_signals = assessment_id if 'assessment_id' in globals() and assessment_id else '1485074653827330048'\n", " model_for_signals = model_id if 'model_id' in globals() and model_id else '1485101794723917824'\n", " signals = []\n", " offset = 0\n", " limit = 10\n", " while True:\n", " params = {'outputType': 'Explanations', 'offset': offset, 'limit': limit}\n", " path = f'/accounts/{ACCOUNT_ID}/datastreams/{ds_for_signals}/assessments/{assess_for_signals}/models/{model_for_signals}/outputsignals'\n", " resp = api_get(path, params=params)\n", " if isinstance(resp, dict) and 'items' in resp:\n", " items = resp['items']\n", " elif isinstance(resp, list):\n", " items = resp\n", " else:\n", " items = []\n", " if not items:\n", " break\n", " for it in items:\n", " sid = it.get('id')\n", " sname = it.get('name') or None\n", " signals.append({'signalId': sid, 'signalName': sname})\n", " offset += limit\n", " df_exp_output_signals = pd.DataFrame(signals)\n", " print('Collected', len(df_exp_output_signals), 'explanation output signals (paged by', limit, ')')\n", " try:\n", " display(df_exp_output_signals.head(20))\n", " except Exception:\n", " print(df_exp_output_signals.head(20).to_string())\n", "except Exception as exc:\n", " print('Failed to collect output signals:', exc)" ] }, { "cell_type": "markdown", "id": "fa77fa16", "metadata": {}, "source": [ "# 6. (OPTIONAL) Page through Prediction output signals (10 at a time) and collect signal id + name" ] }, { "cell_type": "code", "execution_count": 6, "id": "2c1f8726", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collected 1 prediction output signals (paged by 10 )\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "signalId", "rawType": "object", "type": "string" }, { "name": "signalName", "rawType": "object", "type": "string" } ], "ref": "e05d07b1-4286-4bd3-b1f0-455fa1824bcf", "rows": [ [ "0", "1511824332212297728", "Predictions" ] ], "shape": { "columns": 2, "rows": 1 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
signalIdsignalName
01511824332212297728Predictions
\n", "
" ], "text/plain": [ " signalId signalName\n", "0 1511824332212297728 Predictions" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "try:\n", " ds_for_signals = datastream_id if 'datastream_id' in globals() and datastream_id else '1485074653623840768'\n", " assess_for_signals = assessment_id if 'assessment_id' in globals() and assessment_id else '1485074653827330048'\n", " model_for_signals = model_id if 'model_id' in globals() and model_id else '1485101794723917824'\n", " signals = []\n", " offset = 0\n", " limit = 10\n", " while True:\n", " params = {'outputType': 'Predictions', 'offset': offset, 'limit': limit}\n", " path = f'/accounts/{ACCOUNT_ID}/datastreams/{ds_for_signals}/assessments/{assess_for_signals}/models/{model_for_signals}/outputsignals'\n", " resp = api_get(path, params=params)\n", " if isinstance(resp, dict) and 'items' in resp:\n", " items = resp['items']\n", " elif isinstance(resp, list):\n", " items = resp\n", " else:\n", " items = []\n", " if not items:\n", " break\n", " for it in items:\n", " sid = it.get('id')\n", " sname = it.get('name') or None\n", " signals.append({'signalId': sid, 'signalName': sname})\n", " offset += limit\n", " df_pred_output_signals = pd.DataFrame(signals)\n", " print('Collected', len(df_pred_output_signals), 'prediction output signals (paged by', limit, ')')\n", " try:\n", " display(df_pred_output_signals.head(20))\n", " except Exception:\n", " print(df_pred_output_signals.head(20).to_string())\n", "except Exception as exc:\n", " print('Failed to collect output signals:', exc)" ] }, { "cell_type": "markdown", "id": "3f17a0af", "metadata": {}, "source": [ "# 7. (OPTIONAL) Save outputs to CSV files (if available)" ] }, { "cell_type": "code", "execution_count": 7, "id": "9415f4c4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Saved CSV artifacts: events.csv, explanation_output_signals.csv, prediction_output_signals.csv, explanation_signal_names.json\n" ] } ], "source": [ "try:\n", " if EXTRACT_LEARN_EVENTS:\n", " if 'df_events' in globals(): df_events.to_csv('events-learn.csv', index=False)\n", " if 'df_signals' in globals(): df_signals.to_csv('model_signals.csv', index=False)\n", " if 'df_entities' in globals(): df_entities.to_csv('entities.csv', index=False)\n", " if 'df_exp_output_signals' in globals(): df_exp_output_signals.to_csv('explanation_output_signals.csv', index=False)\n", " if 'df_pred_output_signals' in globals(): df_pred_output_signals.to_csv('prediction_output_signals.csv', index=False)\n", " import json\n", " if 'explanation_names' in globals():\n", " with open('explanation_signal_names.json','w') as f:\n", " json.dump(explanation_names,f,indent=2)\n", " print('Saved CSV artifacts: events.csv, explanation_output_signals.csv, prediction_output_signals.csv, explanation_signal_names.json')\n", "except Exception as exc:\n", " print('Could not save outputs:', exc)\n" ] }, { "cell_type": "markdown", "id": "b321ee07", "metadata": {}, "source": [ "# 8. Fetch per-event per-signal explanation cells" ] }, { "cell_type": "code", "execution_count": 8, "id": "a68f68be", "metadata": {}, "outputs": [ { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "event_counter", "rawType": "object", "type": "string" }, { "name": "time", "rawType": "object", "type": "string" }, { "name": "event_label", "rawType": "object", "type": "string" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1", "rawType": "float64", "type": "float" }, { "name": "event_type", "rawType": "object", "type": "string" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Kinetic_Energy", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_HeatFlux", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_Stave_Manifold_Heat_Flux_Total", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_BlastPressureHot", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_EFF", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_FY_401_3", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE_AVG", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Differential_Pressure", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_EquivWindTHM", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Charges_Per_Hour", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO2", "rawType": "float64", "type": "float" }, { "name": "Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO", "rawType": "float64", "type": "float" } ], "ref": "e61569e2-7581-47da-88a2-f0dfc223cc8f", "rows": [ [ "0", "unstable_1", "2026-01-19T13:00:00Z", "unstable", "0.026388786733150482", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "1", "unstable_1", "2026-01-19T13:10:00Z", "unstable", "0.025304947048425674", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "2", "unstable_1", "2026-01-19T13:20:00Z", "unstable", "0.027845997363328934", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "3", "unstable_1", "2026-01-19T13:30:00Z", "unstable", "0.033216629177331924", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "4", "unstable_1", "2026-01-19T13:40:00Z", "unstable", "0.036677420139312744", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "5", "unstable_1", "2026-01-19T13:50:00Z", "unstable", "0.03964701294898987", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "6", "unstable_1", "2026-01-19T14:00:00Z", "unstable", "0.03758585825562477", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "7", "unstable_1", "2026-01-19T14:10:00Z", "unstable", "0.03467544540762901", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "8", "unstable_1", "2026-01-19T14:20:00Z", "unstable", "0.0355636328458786", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "9", "unstable_1", "2026-01-19T14:30:00Z", "unstable", "0.03129621595144272", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "10", "unstable_1", "2026-01-19T14:40:00Z", "unstable", "0.032854292541742325", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "11", "unstable_1", "2026-01-19T14:50:00Z", "unstable", "0.031574998050928116", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "12", "unstable_1", "2026-01-19T15:00:00Z", "unstable", "0.030295705422759056", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "13", "unstable_1", "2026-01-19T15:10:00Z", "unstable", "0.03136824816465378", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "14", "unstable_1", "2026-01-19T15:20:00Z", "unstable", "0.030621930956840515", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "15", "unstable_1", "2026-01-19T15:30:00Z", "unstable", "0.023206379264593124", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "16", "unstable_1", "2026-01-19T15:40:00Z", "unstable", "0.012664718553423882", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "17", "unstable_1", "2026-01-19T15:50:00Z", "unstable", "0.011250670067965984", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "18", "unstable_1", "2026-01-19T16:00:00Z", "unstable", "0.018912404775619507", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "19", "unstable_1", "2026-01-19T16:10:00Z", "unstable", "0.03327150270342827", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "20", "unstable_1", "2026-01-19T16:20:00Z", "unstable", "0.04231470823287964", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "21", "unstable_1", "2026-01-19T16:30:00Z", "unstable", "0.03839442878961563", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "22", "unstable_1", "2026-01-19T16:40:00Z", "unstable", "0.026065843179821968", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "23", "unstable_1", "2026-01-19T16:50:00Z", "unstable", "0.012207791209220886", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "24", "unstable_1", "2026-01-19T17:00:00Z", "unstable", "0.009919744916260242", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "25", "unstable_1", "2026-01-19T17:10:00Z", "unstable", "0.017822347581386566", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "26", "unstable_1", "2026-01-19T17:20:00Z", "unstable", "0.027363266795873642", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "27", "unstable_1", "2026-01-19T17:30:00Z", "unstable", "0.031360119581222534", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "28", "unstable_1", "2026-01-19T17:40:00Z", "unstable", "0.0372161902487278", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "29", "unstable_1", "2026-01-19T17:50:00Z", "unstable", "0.03796932101249695", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "30", "unstable_1", "2026-01-19T18:00:00Z", "unstable", "0.03964701294898987", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "31", "unstable_1", "2026-01-19T18:10:00Z", "unstable", "0.038812436163425446", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "32", "unstable_1", "2026-01-19T18:20:00Z", "unstable", "0.03964701294898987", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "33", "unstable_1", "2026-01-19T18:30:00Z", "unstable", "0.038812436163425446", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "34", "unstable_1", "2026-01-19T18:40:00Z", "unstable", "0.03926318883895874", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "35", "unstable_1", "2026-01-19T18:50:00Z", "unstable", "0.03964701294898987", "learn", null, null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "36", "unstable_1", "2026-01-19T13:00:00Z", "unstable", null, "learn", "0.004367183893918991", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "37", "unstable_1", "2026-01-19T13:10:00Z", "unstable", null, "learn", "0.005002889782190323", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "38", "unstable_1", "2026-01-19T13:20:00Z", "unstable", null, "learn", "0.00738923205062747", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "39", "unstable_1", "2026-01-19T13:30:00Z", "unstable", null, "learn", "0.010890504345297813", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "40", "unstable_1", "2026-01-19T13:40:00Z", "unstable", null, "learn", "0.015755532309412956", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "41", "unstable_1", "2026-01-19T13:50:00Z", "unstable", null, "learn", "0.015377177856862545", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "42", "unstable_1", "2026-01-19T14:00:00Z", "unstable", null, "learn", "0.0146012669429183", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "43", "unstable_1", "2026-01-19T14:10:00Z", "unstable", null, "learn", "0.018286943435668945", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "44", "unstable_1", "2026-01-19T14:20:00Z", "unstable", null, "learn", "0.017702868208289146", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "45", "unstable_1", "2026-01-19T14:30:00Z", "unstable", null, "learn", "0.021225860342383385", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "46", "unstable_1", "2026-01-19T14:40:00Z", "unstable", null, "learn", "0.01987089216709137", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "47", "unstable_1", "2026-01-19T14:50:00Z", "unstable", null, "learn", "0.023713313043117523", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "48", "unstable_1", "2026-01-19T15:00:00Z", "unstable", null, "learn", "0.027300816029310226", null, null, null, null, null, null, null, null, null, null, null, null, null ], [ "49", "unstable_1", "2026-01-19T15:10:00Z", "unstable", null, "learn", "0.027587199583649635", null, null, null, null, null, null, null, null, null, null, null, null, null ] ], "shape": { "columns": 19, "rows": 4560 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
event_countertimeevent_labelExplanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1event_typeExplanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Kinetic_EnergyExplanation-stelco.ca/lewbfprdpi.BF1_HeatFluxExplanation-stelco.ca/lewbfprdpi.BF1_Stave_Manifold_Heat_Flux_TotalExplanation-stelco.ca/lewbfprdpi.BF1_BlastPressureHotExplanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_EFFExplanation-stelco.ca/lewbfprdpi.BF1_FY_401_3Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE_AVGExplanation-stelco.ca/lewbfprdpi.BF1_CALC_Differential_PressureExplanation-stelco.ca/lewbfprdpi.BF1_CALC_EquivWindTHMExplanation-stelco.ca/lewbfprdpi.BF1_CALC_Charges_Per_HourExplanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO2Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO
0unstable_12026-01-19T13:00:00Zunstable0.026389learnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
1unstable_12026-01-19T13:10:00Zunstable0.025305learnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
2unstable_12026-01-19T13:20:00Zunstable0.027846learnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
3unstable_12026-01-19T13:30:00Zunstable0.033217learnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
4unstable_12026-01-19T13:40:00Zunstable0.036677learnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
............................................................
4555unstable_112026-04-25T10:10:00ZunstableNaNlearnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.027508
4556unstable_112026-04-25T10:20:00ZunstableNaNlearnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.027186
4557unstable_112026-04-25T10:30:00ZunstableNaNlearnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.026863
4558unstable_112026-04-25T10:40:00ZunstableNaNlearnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.026863
4559unstable_112026-04-25T10:50:00ZunstableNaNlearnNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.026863
\n", "

4560 rows × 19 columns

\n", "
" ], "text/plain": [ " event_counter time event_label \\\n", "0 unstable_1 2026-01-19T13:00:00Z unstable \n", "1 unstable_1 2026-01-19T13:10:00Z unstable \n", "2 unstable_1 2026-01-19T13:20:00Z unstable \n", "3 unstable_1 2026-01-19T13:30:00Z unstable \n", "4 unstable_1 2026-01-19T13:40:00Z unstable \n", "... ... ... ... \n", "4555 unstable_11 2026-04-25T10:10:00Z unstable \n", "4556 unstable_11 2026-04-25T10:20:00Z unstable \n", "4557 unstable_11 2026-04-25T10:30:00Z unstable \n", "4558 unstable_11 2026-04-25T10:40:00Z unstable \n", "4559 unstable_11 2026-04-25T10:50:00Z unstable \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_1 event_type \\\n", "0 0.026389 learn \n", "1 0.025305 learn \n", "2 0.027846 learn \n", "3 0.033217 learn \n", "4 0.036677 learn \n", "... ... ... \n", "4555 NaN learn \n", "4556 NaN learn \n", "4557 NaN learn \n", "4558 NaN learn \n", "4559 NaN learn \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_2 \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_LIT_759_3 \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_Tuyere_Kinetic_Energy \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_HeatFlux \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_Stave_Manifold_Heat_Flux_Total \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_BlastPressureHot \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_EFF \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_FY_401_3 \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_SOMA_ZONE_AVG \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Differential_Pressure \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_CALC_EquivWindTHM \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_CALC_Charges_Per_Hour \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO2 \\\n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 NaN \n", "4556 NaN \n", "4557 NaN \n", "4558 NaN \n", "4559 NaN \n", "\n", " Explanation-stelco.ca/lewbfprdpi.BF1_FT_Gas_Analyzer2_CO \n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN \n", "4 NaN \n", "... ... \n", "4555 0.027508 \n", "4556 0.027186 \n", "4557 0.026863 \n", "4558 0.026863 \n", "4559 0.026863 \n", "\n", "[4560 rows x 19 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Ensure prerequisites\n", "if 'df_events' not in globals() or 'df_exp_output_signals' not in globals():\n", " raise SystemExit('Missing df_events or df_exp_output_signals; run earlier cells first')\n", "# Resolve entity id if not already set\n", "if not entity_id:\n", " raise SystemExit(f'Entity id for {ENTITY_NAME} not found')\n", "\n", "# Prepare signals list\n", "signals = df_exp_output_signals.to_dict('records')\n", "results = []\n", "row = {} \n", "rows = []\n", "event_counter = 0\n", "events_data = {} # collect all event dataframes in memory for further processing\n", "\n", "# Iterate events and fetch each explanation signal's cells\n", "for _, ev in df_events.sort_values('start').iterrows():\n", " event_counter += 1\n", " label = str(ev.get('label') or 'event').strip()\n", " start = ev.get('start')\n", " end = ev.get('end')\n", " event_type = ev.get('event_type') or 'unknown'\n", " if pd.isna(start) or pd.isna(end):\n", " print(f'Skipping event {label} due to missing start/end')\n", " continue\n", " start_iso = pd.to_datetime(start, utc=True).strftime('%Y-%m-%dT%H:%M:%S.%fZ')\n", " end_iso = pd.to_datetime(end, utc=True).strftime('%Y-%m-%dT%H:%M:%S.%fZ')\n", " time_map = {} # maps time_iso -> { signal_name: value }\n", " for s in signals:\n", " row = {} # reset row for each signal\n", " sid = s.get('signalId')\n", " sname = s.get('signalName') or f'signal_{sid}'\n", " path = f'/accounts/{ACCOUNT_ID}/datastreams/{datastream_id}/assessments/{assessment_id}/models/{model_id}/outputsignals/{sid}/cells'\n", " try:\n", " resp = api_get(path, params={'start_time_iso': start_iso, 'end_time_iso': end_iso, 'level': LEVEL, 'entity': entity_id})\n", " except Exception as exc:\n", " print(f'Failed fetch for signal {sname} ({sid}): {exc}')\n", " continue\n", " if not resp:\n", " continue\n", " # normalize possible response shapes\n", " firstpoint = resp.get('firstpoint') if isinstance(resp, dict) else None\n", " mean = resp.get('mean') if isinstance(resp, dict) else None\n", " if not firstpoint or not mean:\n", " # try alternate keys or list-wrapped responses\n", " if isinstance(resp, list) and resp:\n", " cell = resp[0]\n", " firstpoint = cell.get('firstpoint') or cell.get('firstPoint') or []\n", " mean = cell.get('mean') or cell.get('means') or []\n", " else:\n", " firstpoint = resp.get('firstpoint') or resp.get('firstPoint') or []\n", " mean = resp.get('mean') or resp.get('means') or []\n", " if not firstpoint or not mean:\n", " print(f'No firstpoint/mean for {sname} ({sid})')\n", " continue\n", " for i, t in enumerate(firstpoint):\n", " try:\n", " t_iso = pd.to_datetime(t, utc=True).strftime('%Y-%m-%dT%H:%M:%SZ')\n", " except Exception:\n", " t_iso = str(t)\n", " try:\n", " val = mean[i]\n", " except Exception:\n", " val = None\n", " row['event_counter'] = label + \"_\" + str(event_counter)\n", " row['time'] = t_iso\n", " # record event label and the signal value for this timestamp\n", " row['event_label'] = label\n", " row[sname] = val\n", " row['event_type'] = event_type\n", " rows.append(row.copy())\n", "\n", "df = pd.DataFrame(rows)\n", "df" ] }, { "cell_type": "markdown", "id": "1075f84b", "metadata": {}, "source": [ "# 9. Transpose to desired format and reduce the precision to 3 decimals" ] }, { "cell_type": "code", "execution_count": 9, "id": "41b56db6", "metadata": {}, "outputs": [], "source": [ "# Identify the important signals per event type based on median explanation score\n", "#. Export the \n", "df_out = (\n", " df.rename(columns=lambda c: c.replace(SIGNAL_PREFIX, \"\"))\n", " .groupby(\"event_counter\")\n", " .median(numeric_only=True)\n", " .T\n", " .reset_index()\n", " .rename(columns={\"index\": \"signals\"})\n", " .round(3)\n", ")\n", "\n", "for eoi in EVENTS_OF_INTEREST:\n", " event_cols = [c for c in df_out.columns if eoi in c] # select columns belonging to this event\n", " if not event_cols:\n", " continue\n", "\n", " df_event = df_out.copy() # initialize a fresh dataframe copy\n", " df_event[\"median_exp_score\"] = df_event[event_cols].median(axis=1) # compute median only for the selected columns\n", "\n", " # use in model flag\n", " df_event[\"use in model\"] = df_event[\"median_exp_score\"].apply(\n", " lambda x: \"keep\" if x > EXPLANATION_THRESHOLD else \"remove\"\n", " )\n", "\n", " df_event.rename(columns={\"signals\": f\"{eoi}_signals\"}, inplace=True) # rename signals column dynamically\n", " df_event = df_event[[f\"{eoi}_signals\", \"median_exp_score\", \"use in model\"] + event_cols] # keep only relevant columns\n", " df_event = df_event.sort_values(\"median_exp_score\", ascending=False) # sort descending by median\n", " df_event[\"median_exp_score\"] = df_event[\"median_exp_score\"].round(3) # optional rounding\n", " df_event.to_csv(f\"{eoi}_signal_relevance.csv\", index=False) # export CSV" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 5 }