-
Notifications
You must be signed in to change notification settings - Fork 1
Questions generator & validator #446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kiyro7
wants to merge
109
commits into
master
Choose a base branch
from
questions_generator
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
109 commits
Select commit
Hold shift + click to select a range
33b8189
add test endpoint
Shumer-1 328972f
added stub
Shumer-1 28220b6
added new endpoint and question collection
Shumer-1 2832d83
removed unused parts
Shumer-1 835b5ce
initial commit
kiyro7 39d54cf
first prototype
kiyro7 d813c88
added LLM questions marker
kiyro7 f6e00b6
fix avatar uploading and models
Shumer-1 8e42c8e
removed methodology
kiyro7 48ed43c
requirements.txt added versions
kiyro7 1bcf046
simplified docker
kiyro7 8a54af1
heuristic patterns update
kiyro7 d7a57d7
updated questions ranking and added examples
kiyro7 e20a3e0
docker-compose finally done
kiyro7 5694ae7
interactive mode
kiyro7 6ec4877
logging added
kiyro7 0b28da7
logging update
kiyro7 39f7626
docker fix (builds aprox 40 mins)
kiyro7 bee9a7a
fixed heuristic questions generation
kiyro7 fa52f19
added front and states
Shumer-1 a16784b
clearing
kiyro7 c2df6e4
created static folder
kiyro7 666535d
full logs refactor and translation to russian
kiyro7 e92b6ac
stashed new question generator code for future updates
kiyro7 e7e72da
docker update - llm & stuff and code separated
kiyro7 21b960b
global question generator refactor
kiyro7 a89eb4d
added instructions
kiyro7 72916c8
add getting questions from database
Shumer-1 dc1cbd4
docx_parser from document_insight_system prototype (works) & docker f…
kiyro7 9adab60
testing paragraphs max nesting (founded max depth - 1)
kiyro7 7db5faf
first prototype of chapters detection (works)
kiyro7 755e248
add saving answers
Shumer-1 898ee3e
fix lti
Shumer-1 3e7cb1c
generator refactor for usage of DocxUploader & docker llm-init servic…
kiyro7 2c595cb
reduced num_beams for model and shuffled chunks of text for questions…
kiyro7 8869223
Merge pull request #448 from OSLL/test_endpoint
kiyro7 3cd08d7
Merge branch 'master' into questions_generator
kiyro7 9c8122f
added celery
kiyro7 9cadc4d
compose fix
kiyro7 f3fc98e
add answer evaluation
Shumer-1 38e3128
add result page
Shumer-1 040c7f2
fix docker
Shumer-1 83c77f2
possible compose fix
kiyro7 0b9cfd2
possible compose fix #2
kiyro7 1fb009d
CELERY WORKS!!!
kiyro7 fe7c97c
add upload page
Shumer-1 b20e2ba
add celery task logic
Shumer-1 1c1da23
fix celery
Shumer-1 ab80707
add celery logic
Shumer-1 c9f5dbd
Merge pull request #450 from OSLL/test_endpoint
kiyro7 d1d807b
first fixes
kiyro7 7b68940
dockerfiles fixes
kiyro7 95c1c1b
requirements.txt fix
kiyro7 c8464d4
questions order shuffle
kiyro7 e302609
add cancel session logic
Shumer-1 db72c5e
fix auth logic and add show all interview
Shumer-1 7d89bf0
some fix saving
Shumer-1 e3b1742
refactor code
Shumer-1 db8302f
add vars to config
Shumer-1 9ba6b53
add response
Shumer-1 1251040
fix window logic
Shumer-1 d8658ee
use format file backend logic
Shumer-1 1f8bb1c
fixes (Docker, generator)
kiyro7 9961141
Merge remote-tracking branch 'origin/questions_generator' into questi…
kiyro7 fc4cb45
mongo version update
kiyro7 f3bde7b
Revert "use format file backend logic"
Shumer-1 99ba79a
Revert "fix window logic"
Shumer-1 47c74a1
fix format logic
Shumer-1 1e9560a
add document ready and update config
Shumer-1 501f750
fix window logic
Shumer-1 b619bcf
fix upload page
Shumer-1 05ba373
fix result page
Shumer-1 7aea665
add new mongo odm file
Shumer-1 8f36341
fix task service
Shumer-1 9abaa1a
remove unused config
Shumer-1 b973df1
move methods
Shumer-1 daae10b
Docker simplified & generator fixes
kiyro7 3c0cdcf
Merge remote-tracking branch 'origin/questions_generator' into questi…
kiyro7 376cef4
llm questions fix & DbManager fix
kiyro7 0358789
add some fix
Shumer-1 5372f0c
add new criteria
Shumer-1 d459e64
add time limit
Shumer-1 1774c05
some fix
Shumer-1 97f934f
add json
Shumer-1 86b4741
celery config added
kiyro7 06b23c6
moved nltk check
kiyro7 896396d
first validator fixes
kiyro7 7ec4309
celery config added
kiyro7 44da30b
moved nltk check
kiyro7 1e14a02
first validator fixes
kiyro7 d823dfd
Merge branch 'questions_generator' of github.com:OSLL/web_speech_trai…
kiyro7 d375dcb
validator_constants.py added & nltk download moved to Dockerfile
kiyro7 a737062
fix compose
Shumer-1 b6317e8
fix compose
Shumer-1 2f18e8f
fix compose
Shumer-1 b4af209
fix dockerfile
Shumer-1 0cfebbd
fix dockerfile_base
Shumer-1 1d24363
fix dockerfile_base
Shumer-1 f9cf416
fix compose and add profile
Shumer-1 1867f67
logger fix
kiyro7 8233cf9
remove unused window
Shumer-1 257681e
added logger for research purposes, logs in volume
kiyro7 48e750a
add variables to lti and update readme
Shumer-1 8e3b270
fix pause criteria
Shumer-1 7a9496e
add filter
Shumer-1 c84c9c9
return logger
Shumer-1 9ba80ac
add attempt limit
Shumer-1 35af4cc
fix interview
Shumer-1 5473a38
add logging
Shumer-1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,16 +1,30 @@ | ||
| FROM ubuntu:22.04 | ||
| ENV LANG C.UTF-8 | ||
| ENV LANG=C.UTF-8 \ | ||
| DEBIAN_FRONTEND=noninteractive \ | ||
| PIP_DEFAULT_TIMEOUT=120 \ | ||
| PIP_RETRIES=10 \ | ||
| PIP_DISABLE_PIP_VERSION_CHECK=1 \ | ||
| PIP_INDEX_URL=https://pypi.org/simple | ||
|
|
||
| LABEL version="0.2" | ||
| LABEL version="0.3" | ||
| LABEL project="wst" | ||
|
|
||
| RUN apt update && apt install -y software-properties-common | ||
| RUN add-apt-repository ppa:libreoffice/ppa && apt update | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| python3-pip=23.0.1+dfsg-1 \ | ||
| python3-dev=3.11.2-1+b1 \ | ||
| ffmpeg=7:5.1.8-0+deb12u1 \ | ||
| libimage-exiftool-perl=12.57+dfsg-1 \ | ||
| inkscape=1.2.2-2+b1 \ | ||
| mupdf=1.21.1+ds2-1+deb12u1 \ | ||
| mupdf-tools=1.21.1+ds2-1+deb12u1 \ | ||
| libgconf-2-4=3.2.6-8 \ | ||
| libnss3=2:3.87.1-1+deb12u2 \ | ||
| libxss1=1:1.2.3-1 \ | ||
| libmagic1=1:5.44-3 \ | ||
| nano=7.2-1+deb12u1 \ | ||
| libreoffice-impress=4:7.4.7-1+deb12u10 \ | ||
| default-jre=2:1.17-74 \ | ||
| ca-certificates=20230311+deb12u1 && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| RUN apt install -y --no-install-recommends libgconf-2-4 libnss3 libxss1 libmagic1 python3-pip python3-dev ffmpeg exiftool inkscape mupdf mupdf-tools libmagic1 \ | ||
| nano libreoffice-impress default-jre | ||
|
|
||
| RUN pip3 install --upgrade pip | ||
|
|
||
| COPY requirements.txt requirements.txt | ||
| RUN pip3 install --ignore-installed --no-cache-dir -r requirements.txt | ||
| COPY requirements.txt ./ | ||
| RUN python3 -m pip install --no-cache-dir -r requirements.txt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,6 @@ | ||
| # web_speech_trainer | ||
| # web_speech_trainer | ||
|
|
||
| ## Инструкция по полному и частичному запуску приложения | ||
| - `docker-compose --profile training_profile --profile interview_profile --profile question_generate_profile up --build` - запуск приложения полностью (оба режима) | ||
| - `docker-compose up --profile training_profile` - запуск приложения без функционала интервью | ||
| - `docker-compose up --profile interview_profile` - запуск приложения без функционала тренировок |
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
а как мы его будем обновлять / использовать, если он в игноре?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
здесь в гитигнор добавлена папка с примерами файлов ВКР, которые использовались при разработке. не вижу смысла держать их на гите (как минимум из соображения персональных данных), но можно убрать если нужно
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
сейчас снова посмотрел, возможно вы про /whisper_asr_model_cache - он даже сейчас на мейне в гитигноре