Use multiprocessing in machine learning (pegasus_paraphrase)

I have seen myself use this script that paraphrases sentences of 60 characters: [login to view URL], if that short the text works fine, but if I have a text of 1000 words it takes me between 2 to 4 minutes to paraphrase it. So not if you could help me to return the result faster, I was reading about multiprocessing but I do not see improvement, I also tried putting num_proc = 12 in tokenizer but it does not go faster, I have an environment of the following characteristics:

12 Dedicated Cores

16G Dedicated RAM

50GB NVMe SSD Space

Dedicated Gigabit Network

My code in Python:

import torch

import flask

import json

from multiprocess import Pool

from sentence_splitter import SentenceSplitter, split_text_into_sentences

from transformers import PegasusForConditionalGeneration, PegasusTokenizer

from flask import request, jsonify

model_name = 'tuner007/pegasus_paraphrase'

torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'

tokenizer = PegasusTokenizer.from_pretrained(model_name)

model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)

app = [login to view URL](__name__)

[login to view URL]["DEBUG"] = True

def get_response(input_text):

batch = tokenizer([input_text],truncation=True,padding='longest',num_proc=7, max_length=60, return_tensors="pt").to(torch_device)

translated = [login to view URL](**batch,max_length=60,num_beams=10, num_return_sequences=1, temperature=1.5)

tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)

return tgt_text

@[login to view URL]('/api/rewrite', methods=['POST'])

def home():

#data = request.get_json(force=True, silent=True)

#args = parser.parse_args()

#context = [login to view URL]

splitter = SentenceSplitter(language='en')

sentence_list = [login to view URL]([login to view URL]['text'])

paraphrase = []

for i in sentence_list:

result = get_response(i)

[login to view URL](result)

paraphrase2 = [' '.join(x) for x in paraphrase]

paraphrase3 = [' '.join(x for x in paraphrase2) ]

paraphrased_text = str(paraphrase3).strip('[]').strip("'")

return { "result": paraphrased_text }

[login to view URL](host='', port=5027)

Evner: Python, Software Arkitektur, Machine Learning (ML)

Se mere: machine learning use cases in networking, which machine learning algorithm should i use, how to use machine learning in website, machine learning in business use cases, amazon machine learning use cases, use of machine learning in software testing, how to use google cloud machine learning, how to use machine learning in web application, plagiarism detection using machine learning-based paraphrase recognizer, machine learning use cases, telecom machine learning use cases, use of statistics in machine learning, when to use deep learning vs machine learning, sap machine learning use cases, how to use machine learning in django, what machine learning algorithm you ll use to avoid spam comments, machine learning use cases in life sciences, use of machine learning approaches for novel drug discovery., how to use a machine learning model, which of the following is not a current use of machine learning in healthcare

Om arbejdsgiveren:
( 2 bedømmelser ) Breña, Lima, Peru

Projekt ID: #31586578

3 freelancere byder i gennemsnit $43 timen for dette job


Hi, i would like to complete this work for you. I have experience working with pytorch and flask api and i would provide you very good service. Thanks.

$50 USD in 7 dage
(4 bedømmelser)

Hi, We can do this task earlier we did this type work for Omani Client for Arabic, if you provide data we can give a try. Thanks

$30 USD in 7 dage
(1 bedømmelse)

Hello, there My name is Francesco Schuster and I may have found a solution to your problem using python threads and the threading library. It involves creating a new thread for each element in sentence_list. Running ea Flere

$50 USD in 5 dage
(0 bedømmelser)