Releasing train & test datasets (2024. 08. 12)

<aside> 💡 Releasing train & test datasets (2024. 08. 12)

All training and test datasets for the challenge have been released.
The challenge is organized into Track 1 and Track 2, and participants can choose to participate in one or both tracks.
For more details, please refer to the section Dataset Composition and Download.
The final submission deadline is September 20th.

2024 KSNVE AI Challenge - Google Drive

</aside>

Urgent Notice (2024. 08. 09)

<aside> 💡 Urgent Notice (2024. 08. 09)

Errors have been found in the dataset used for the challenge, where excessive noise has been added, making SNRs significantly lower than intended.

Therefore, we plan to redistribute the corrected training, validation, and evaluation datasets on August 12th.

Due to insufficient time for developing with new data, the final submission deadline for the challenge has been extended from August 30th to September 20th.

Please refer to the new schedule shown below for more detailed information.

We sincerely apologize for the inconvenience this has caused all participants.

</aside>

(New) 2-Track Challenge

<aside> 💡 For the participants who already secured good performance on the previous dataset, we also operate a separate challenge track.

[Challenge 1] Normal data track: A challenge track using the data to be redistributed on August 12th.
**[Challenge 2]** **Crazy data track**: A challenge track using the data released on May 18th.
Participants can choose to participate in one of the two challenges or submit entries for both tracks.
We plan to award prizes for both tracks. </aside>

Schedule (2024)

| May 18 | Challenge launch (Release of datasets and baseline model) | | --- | --- | | May 23 | Challenge presentation (KSNVE Spring Conference) | | August 12 | Evaluation & New Development datasets release | | September 20 | Challenge deadline (21:00) | | October 4 | Team rankings announcement | | October 24 | Challenge winning team award ceremony (KSNVE Autumn Conference) |

Anomaly Detection Challenge Using Speed Variable Bearing Vibration Dataset

<aside> 💡 Table of Contents

</aside>

<aside> 📌 Short-cut

❗(new) Pre-registration

KSNVE 2024 AI Challenge Baseline

Data Download Site

**Submission Site**

</aside>

1. Overview

The rotating parts of mechanical equipment operate under various speed conditions, and detecting their faults can often be challenging.

In particular, rolling bearings, which are essential components of rotating machinery, can be damaged by various causes such as faults in the inner or outer rings and damage to the balls. It is necessary to detect any abnormalities in advance.

The 2024 Korean Society of Noise and Vibration Engineering AI Challenge (2024 KSNVE AI challenge) aims to detect anomalous rolling bearings using deep learning and a vibration dataset of rolling bearings measured under variable speed conditions (680 RPM ~ 2460 RPM).

2. Goals

The final goal of this challenge is to develop an anomaly detection model capable of identifying the presence or absence of abnormalities in bearings. This model will utilize vibration data collected under variable speed conditions.

Participants should train an AI model using training rolling bearing vibration data and then effectively detect the presence of bearing faults using the provided test data.
Considering the scarcity of defective bearing data in practical conditions, the AI model should be trained exclusively with normal bearing vibration data (self-supervised learning).
After training, participants should generate and submit an anomaly detection score for each test data, which includes both abnormal and normal bearing data.

3. Datasets

3.1 Measurement methods for bearing vibration signals

The testbed for the rotating machine includes a three-phase induction motor, a torque meter, a gearbox, bearing housings A and B, rotors, and a hysteresis brake (Figure 1).
The rolling bearing used is a 6205 steel NSK ball bearing, consisting of an outer ring, an inner ring, a ball, and a cage (Figure 2).
The bearing's surface was artificially damaged with a diamond tip to simulate four different states of a rolling bearing, as shown in Figure 3 (normal state - normal, inner race fault - inner, outer race fault - outer, ball fault - ball).
The variable speed conditions were implemented by adjusting the rotation frequency of the main motor.
The vibration data were measured in the x and y directions of the bearing housing using a PCB 352C34 accelerometer.
A Siemens SCADAS Mobile 5PM50 measured the data with a sampling frequency of 25.6 kHz.
The dataset consists of vibration data obtained from rolling bearings with various fault types such as different inner or outer race faults, and ball faults while the motor speed conditions varied randomly (680 RPM ~ 2460 RPM).

Figure 1. Layout of the rotating machine testbed

Fig. 2. Description of rolling element structure

(a)

(c)

(b)

(d)

Figure 3. The conditions of bearings: (a) normal state, (b) inner race fault, (c) outer race fault, (d) ball fualt

3.2 Dataset composition and download

Dataset composition (common to Track 1 & 2)

The vibration data for development( train and eval ) and submission (test) are provided.

<aside> 💡 Participants should not use eval and test datasets for training their models.

</aside>

The train dataset includes only the normal bearing data as .csv files.
The eval dataset is provided to assess the performance of the developed model and includes .csv files for each condition (normal, inner, outer, ball) of the bearings.
The test dataset, along with eval dataset, is for competition submissions only, and you should only use it to perform anomaly detection using the model you trained and submit your results.
Each .csv file contains 1 second of data and includes two columns including acceleration data for x- and y- directions.
The unit of the vibration data is the gravitational constant $g$ ($1g = 9.80665 \ m/s^2$).

Dataset download

2024 KSNVE AI Challenge - Google Drive

(1) Track 1 (Normal Data Track)

This challenge track is conducted using the newly distributed dataset on August 12th.
It includes noise at normal levels.

[Dataset Specification (Track 1)](https://ksnve.notion.site/Dataset-Specification-Track-1-bc5e6ef97a884a10ad2a5cb6cc755b61)

(2) Track 2 (Crazy Data Track)

This challenge track is conducted using the initial dataset distributed on May 18th.
(Caution): It contains very high levels of noise, so participation in the Normal Data Track (Track 1) is recommended.

[Dataset Specification (Track 2)](https://ksnve.notion.site/Dataset-Specification-Track-2-4189aa609d8f44a3bceaafcf59a0784d)

4. Baseline Model

This challenge provides a baseline model as an example (description and download link).
The baseline model is a spectrogram-based autoencoder model composed of linear layers.
There is no need to use or improve the baseline model for the challenge. The team’s own model can be used.

5. Anomaly Detection Test and Evaluation Metrics

Test dataset

The test data will be released on August 9th for a fair comparison of the models developed by the participants.

The test data are 6,331 .csv files without labels.

Each .csv file contains 1 second of data like train data

Participants must input eval and test data into the trained model and submit the results. The results should include an Anomaly Score for each input data set, indicating the degree of anomaly.
There are no restrictions on the range of Anomaly Scores. However, the scores should be structured so that lower numbers indicate normal conditions and higher numbers indicate anomalies.

Evaluation Metric

The evaluation metric for the model is ROC-AUC (AU-ROC). ROC-AUC is a metric evaluating how well the distributions of normal or abnormal data are separated, which is independent of the decision boundary used to distinguish between normal and abnormal conditions.

ROC-AUC is calculated by the organizing committee using the .csv file submitted by the participants (no need for calculation and submission by participants).

Figure 4. Evaluation metric (ROC-AUC)

6. Registration and Submission

<aside> 📌 Submission requirement

Anomaly Score for Evaluation, Test data (eval_score.csv, test_score.csv)
Code that can reproduce training and evaluation
Technical report (.pdf) </aside>

This challenge evaluates the performance of the models based on the performance over the eval (eval_score.csv) and test (test_score.csv) data set (Figure 5).
Participants should submit the anomaly score files extracted by eval.py and test.py with the trained model.
Participants must submit the output value (anomaly score) of the model for the eval and test data (Figure 5).

Figure 5. Submission procedure

The reproducible code for the training and evaluation should be submitted. Running the submitted code should reproduce the scores similar to the submitted ones.
The technical report should include a brief description of the network architecture, training method, and anomaly detection method, in a freely formatted document of 1—2 pages.
The submission format for the anomaly score should be a .csv file with a 6,331 × 2 matrix. The first and second columns should include the [File name] of the test or eval data and the score value for each file, respectively (Figure 6).
Upload the following four types of files to the google form.

Figure 6. Example of the result CSV file

<aside> 💡 Please contact [email protected] if you have any questions about this challenge.

</aside>

Untitled