Just to let you know: I've just published a flow with HTTP endpoints for text completion and prompt tokenization Stability AI's StableCode-Completion-Alpha-3B-4K model using ggml under the hood for inferencing on one's own machine.
Further details and instructions can be found in the related GitHub repository .
What does this do - in muggle speak please?
well, in the extreme case, this lets your computer write code in various programming languages for you - and since it offers HTML endpoints, you may access this functionality from a browser or any program that can handle HTTP requests. That way, you may build your own "autonomous agent" - with all components running locally, keeping your data safe and not requiring you to pay OpenAI...
How about in the simplest case?
Or is this something too clever to have a simple use?
well, just tell it to program a loop or s.th. similar - complexity depends on your request (and the abilities of the model - don't expect ChatGPT performance from a 3B LLM). Since everything is free, you may experiment freely with the tool - a user interface is in the making...
Just for comparison: ChatGPT is told to have 1000B parameters, Google Bard around 137B