Do you mean that your toy's time changes by itself? In which case - yes, that's bad.
Or, do you mean that you adjust the time often. In which case - no, it's not bad. The only drawback to doing this is that you may have trouble keeping track of when it's scheduled to evolve (if you care about such matters.)