Occasional latency between the strokes/input signal

hi guys,
occasionally, i’m experiencing a strange problem.
i’m doing a gig (or practising/experimenting at home) and everything is ok. but maybe after half an hour, or after one hour (or never, it’s not something that always happen) suddently i hear a small “crack” sound and the software starts to receive my acoustic input with quite a big latency (200/300 ms, more or less). i mean, when i hit the drum i can clearly see the meter of the channel of the drum raise with this huge latency.
when it happens, the only way to solve the problem is to quit and re-open the software. after that everything come back to normality.
and, even if sometimes the effect is really interesting (because it works like a sort of “thick delay between my frasing and the audio output”), it became a problem because is not predictable and not controllable.
anybody had the same problem?
any solutions?