The limitation is memory. Global and other context variables have to live in memory. I have at least 1 global with well over 10,000 entries though each entry is only a couple of properties.
That wouldn't help. The only thing that would help would be to query the DB each time with a filter to reduce the number of entries. e.g. making the DB do all the work which is typically more efficient.
But with that number, you will probably be fine. Just be careful not to dump the whole thing to debug as that can be slow.
I've thought about making each hit query to the DB, and then only valid results get cached locally, kind of like an intermediate with a fallback to the database.