Schengen rules kind of dance around this issue. The Schengen acquis does not create or remove any specific requirement to hold documents to cross internal borders. It also explicitly outlaws border control (with a few nuances and caveats) so if holding a specific document was required, that requirement could not legally be enforced, at least not systematically.
In fact, I suspect that historically in many places, what's illegal is crossing outside of an official border check point. And if you cannot identify yourself at an official border checkpoint, you would typically be turned away, making the question moot. In these circumstances, even where there is an administrative requirement to hold some specific document, it's not necessarily a separate criminal offence to cross without it but rather something that was meant to be checked by border guards.
At the same time, many Schengen countries (including, it seems, Austria) have strict ID and registration requirements even while residing in the country. The Schengen Borders Code explicitly allows that in article 23(c) and 23(d). This makes the question rather irrelevant in another way: If you come to the attention of the police and they really want to create trouble for you, you are just as likely to be held, fined, or prosecuted under these laws than punished specifically for having crossed the border.
By contrast, in a country where there is no such requirement (e.g. France), you would not be committing a specific offence by crossing the border without your passport and I don't see anything illegal about it. Finally, note that proximity to a border can be legitimate grounds for checks that would otherwise be technically illegal (it's explicitly the case in France and I think in Germany too). So actually crossing the border might not even be required to give the police cover to ask you to identify yourself and ultimately book you for not having your passport on your person (where required) or even a completely unrelated offence stemming from that check.
