[Draft] Add Web API for MarkItDown#202
Conversation
Related to microsoft#133
|
In Testing |
vs4vijay
changed the title
|
Thank you very much, we need api interface access in order to utilize automated processes, we are very optimistic about this project, thanks again to the great developers. |
|
I deployed a simple REST API, ya'll can try it out. Here are the specs: |
GerardSmit
left a comment
GerardSmit
left a comment
There was a problem hiding this comment.
I tried to run the API locally through Docker. I had to change the following things to make it work.
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| RUN pip install markitdown | ||
| RUN pip install markitdown fastapi uvicorn |
There was a problem hiding this comment.
| RUN pip install markitdown fastapi uvicorn | |
| RUN pip install markitdown fastapi uvicorn python-multipart |
Without python-multipart the API won't start.
It's still weird that markitdown is being installed here. This means if the source gets changed and you want to check if the Docker image works you first need to publish the package to pip... It would be better if the current src folder would be included here.
There was a problem hiding this comment.
Use pip install "fastapi[standard]"
When you install with
pip install "fastapi[standard]"it comes with some default optional standard dependencies.If you don't want to have those optional dependencies, you can instead install pip install fastapi.
| USER $USERID:$GROUPID | ||
|
|
||
| ENTRYPOINT [ "markitdown" ] | ||
| ENTRYPOINT ["uvicorn", "src.markitdown.api:app", "--host", "0.0.0.0", "--port", "8000"] |
There was a problem hiding this comment.
This doesn't work: src/markitdown/api.py doesn't exists, so the API will not start.
You must copy the src directory:
| ENTRYPOINT ["uvicorn", "src.markitdown.api:app", "--host", "0.0.0.0", "--port", "8000"] | |
| COPY src /src | |
| ENTRYPOINT ["uvicorn", "src.markitdown.api:app", "--host", "0.0.0.0", "--port", "8000"] |
Note that in .dockerignore that everything is ignored, so the COPY won't work.
Modify the .dockerignore so the src/-folder gets included:
*
!/src
|
I tried out this branch and found a couple of issues; locally I fixed them and rebased the branch on top of the current |
| temp_file.write(contents) | ||
|
|
||
| markitdown = MarkItDown() | ||
| result = markitdown.convert(temp_file_path) |
There was a problem hiding this comment.
when this is a long CPU-bound operation (for example when converting a long PDF) this can block the executor
|
|
||
| try: | ||
| contents = await file.read() | ||
| temp_file_path = f"/tmp/{file.filename}" |
There was a problem hiding this comment.
if multiple requests are accepted concurrently with the same filename, they can conflict
(nb: AFAICT this currently does not occur because requests are handled sequentially)
|
https://github.com/Ahoo-Wang/markitdown/tree/dev/packages/markitdown-api https://github.com/Ahoo-Wang/markitdown/blob/dev/packages/markitdown-api/delpoy/markitdown-api.yml Docker Image:
|
7 participants
Related to #133