Commit Graph

28 Commits

Author SHA1 Message Date
Richard Wong 1b6659a600 Feat: reworked overall evaluation to evaluate at each stage of the
pipeline
2024-12-25 20:35:28 +09:00
Richard Wong 086b867d91 Feat: added overall section to evaluate combined accuracy
- added relevant-class section
2024-12-24 21:57:48 +09:00
Richard Wong 6072e4408c Feat: added modified layer-size decoder variants
- added frozen encoder/decoder variants
2024-12-23 07:36:06 +09:00
Richard Wong c5760d127d Feat: added post_processing based on rules
others:
- added basic data analysis to get histograms of text differences
- added new final delivery model
2024-12-18 13:43:56 +09:00
Richard Wong 481bcf88b7 Feat: added embedding plot for coarse and fine-grained labels 2024-12-12 22:06:26 +09:00
Richard Wong c64e4bccfc Feat: added embedding plots viewer for different models 2024-12-12 16:13:47 +09:00
Richard Wong b01ca4f395 Feat: implement hybrid fine-tuning of encoder and decoder networks
separately
2024-12-10 23:40:10 +09:00
Richard Wong 446ed1429c feat: end-to-end code needed for deployment that includes preprocess,
mapping, post-process (de-duplication)
2024-12-02 14:57:03 +09:00
Richard Wong 737c86bc2e Feat: added de_duplication post-processing method 2024-11-28 11:02:22 +09:00
Richard Wong 8dba46ded6 Chore: removed unnecessary output files 2024-11-25 18:19:52 +09:00
Richard Wong ff6e11a3c0 Feat: added more classification and mapping variations
Feat: added grid-search for threshold in similarity-classifier
Feat: added more abbreviation rules
2024-11-25 18:15:28 +09:00
Richard Wong 1f3970459f Chore: re-organized train folders to have standardized naming schemes
Feat: introduced BERT-based binary classification
2024-11-20 15:07:47 +09:00
Richard Wong 96e7394c59 Feat: tuned selection_with_pattern to perform better 2024-11-11 21:47:24 +09:00
Richard Wong 7699201cb8 Feat: implement selection for pattern-mapping
Feat: added error analysis for BERT find-back
Feat: added direct mapping with unit
Feat: added BERT for classification using description only
2024-11-11 20:20:43 +09:00
Richard Wong bb3ddfaa2f Feat: include basic ood similarity analysis using bert 2024-11-11 02:18:57 +09:00
Richard Wong 2b5994cb52 Feat: added abbreviation expansion rules 2024-11-10 20:28:47 +09:00
Richard Wong 59bbf1f403 Feat: implement find-back for analysis in find_closest.py
Feat: implement bert classification
2024-11-08 20:50:41 +09:00
Richard Wong 22429ea536 Feat: added classification methods
Feat: added mapping to pattern-only method
Chore: re-organized prediction to be within mapping folders
2024-11-05 16:49:18 +09:00
Richard Wong 0ad182f2b9 Feat: added classification for all data, including non-mdm, as a
training baseline model
2024-11-01 13:21:12 +09:00
Richard Wong 0228c5c0fd Doc: updated README.md to reflect execution order 2024-10-31 16:51:47 +09:00
Richard Wong 18e4a5f7df Chore: moved selection to post_process, mapping to test 2024-10-31 16:35:28 +09:00
Richard Wong 16374b9ab8 Feat: added train and test directories 2024-10-31 15:58:20 +09:00
Richard Wong c7a02c792c Feat: added Daniel's abbreviations preprocessing to preprocessing
methods
2024-10-30 11:01:57 +09:00
Richard Wong 4715999005 Chore: changed ipynb to py files in the data_preprocess folder
Doc: added descriptions and instructions for the data_preprocess folder
2024-10-29 22:55:22 +09:00
Richard Wong 67f3712ea6 Chore: re-organized data_import directory to use .py files
Doc: added README.md explaining purpose of each file and instructions
2024-10-29 20:07:51 +09:00
hhs0625 24829c7abf [TASK] the entier paper work 2024-09-25 08:52:30 +09:00
hhs0625 3d2266cf65 [TASK] init 2024-08-26 19:51:11 +09:00
hhs0625 3841867c4b
Initial commit 2024-08-22 20:03:41 +09:00