2024 04¶
4)¶
I am looking for a general strategy how to continue my masters-thesis. The following two questions are guidelines:
llm-resource-physics-knowledge-extraction-question I have finished a simple subtopic network generator:
5)¶
The results of the subtopic network should be evaluated. One possibility of how to proceed would be to create a list of physics terms and then write an algorithm to classify them into a subtopic tree.
8)¶
Explored possibility to host large database files with zenodo Other option is to create github repository for that purpose: https://github.com/gratach/master-database-files
11)¶
Created a subtopic network search project. This project implements a evaluation method for subtopic networks. The next steps could be to apply this evaluation method to different types of large language models. Also the stability of a single search could be tested.
15)¶
Found a solution for the calculation of the statistical error of the percentage of found technical physics terms: https://de.wikipedia.org/wiki/Binomialverteilung#M%C3%BCnzwurf The error calculation of the average search path length is done like described in this link
16)¶
Implemented the subtopic network search evaluation for the mistral model and the cosmosage model of blablador. The cosmosage model is so bad at formatting the output correctly so a post processing with mistral was necessary.
17)¶
Regenerated subtopic network search data for the models gpt-3.5-turbo, gpt-4-turbo, mistral_7b_instruct_v_0_2 and cosmosage with a slightly changed query where I did not mention that I wanted to have the output formated as json because gpt-4-turbo was misinterpreting that information. Temperature change to 1 of model cosmosage leads to notable worse performance so I left the temperature at the default value. Compared models in table.
19)¶
Created a Project for subtopic tree generation which is adapted to a specific list of technical terms.
20)¶
Translated the forum-concept-draft into English
22)¶
Evaluated the results from the adapted subtopic tree generation
24)¶
Created metric for the measurement of the quality of a subtopic tree
25)¶
Added the metric of the viewed bytes per search.
28)¶
Described approach of how to continue with the creation of diverse semantic connections in-between existing physics terms