Hiho,
I ve a project where I need to evaluate new articles by any LLM like Chat GPT. I knew the link to a new article most likely 2000 to 6000 words (2 - 6 Articles per day). So I can try to grab it by html node. And send this grabbed html with a prompt to any LLM. The questions I ve are:
Do you know which LLM cand handle this and cheap or free ?
How about the plan with grabbing it ? - Would you do other ?
Some sites may block it ? - How could I find out if the article was grabbed or is incomplete for some reasons ?
I think most if not all of the API's on the big LLM's cost money. I've never really managed to work out the cost though as the calculations seem rather complex. I'd be interested though if you manage to work out some costs.
Bit hard to say as you haven't really given much to go on. Do remember though that LLM's are totally dependent on their inputs and if you are trying to summarise articles involving subject areas where there is limited input, they will just make stuff up. You need to allow for that.
Each API will have terms and conditions of use, you will need to check them and make sure you don't break them.
If you have a PC with a decent GPU or an Apple silicon based Mac, you can do this locally, a model like llama3.1 has a context window of 128k tokens, 6000 words should be easily feasible together with software like ollama that provides an API. But also note that there are tools that can also include urls to crawl and perform the actions on it directly. I am using open-webui, which has search connectors for urls built-in.
