So, it turned out issue with extrapolation was actually pretty simple :p
Instead of replacing current values (of velocity, rotation etc.) I added calculated values to current one. Calculated values should take into account most recently available server state and not base on client values, since they may be invalid (especially when connection is too slow).