If I hide a field on a public form, can someone still set it?

No. A public form only asks for the fields you choose for it. If someone guesses the name of another field and sends a value for it anyway, the server ignores it. Leave a field off the form and strangers cannot set it.

If a field holds something private, also keep it out of view with access rules. Leaving it off a form stops people from setting it, but it does not stop people on your team from seeing it.