Discussion:
[Spice-devel] spice performance tweaking
Rob Verduijn
2016-09-13 06:47:18 UTC
Permalink
Hello,

I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.

It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.

I have full access to the kvm host, kvm guest and openvpn server.

Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7

Any tips for :
- the KVM host ?
- the KVM guest ?
- the openvpn server ?

Cheers
Rob Verduijn
Christophe Fergeau
2016-09-13 14:43:54 UTC
Permalink
Hey,
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
I would try to minimize latency, and to get as much bandwidth as
possible, probably some tweaks can be done for that on the openvpn side?
Regarding QEMU, make sure you are using the QXL video driver as well as
the SPICE agent in the guest. There are also a bunch of compression
options that you can try to tweak, see
https://libvirt.org/formatdomain.html#elementsGraphics

Christophe
Rob Verduijn
2016-09-13 18:04:04 UTC
Permalink
Post by Christophe Fergeau
Hey,
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and
Post by Rob Verduijn
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
I would try to minimize latency, and to get as much bandwidth as
possible, probably some tweaks can be done for that on the openvpn side?
Regarding QEMU, make sure you are using the QXL video driver as well as
the SPICE agent in the guest. There are also a bunch of compression
options that you can try to tweak, see
https://libvirt.org/formatdomain.html#elementsGraphics
Christophe
bandwith and latency are not mine to decide about
QXL-driver check
spice-agent check
tweaks in guest definition :
<graphics type='spice' port='5999' autoport='no' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
<image compression='glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>
</graphics>

I tried them on auto_glz,auto,auto,filter (tried every combo)
But I guess my bandwith is not stable and since setting them like this
gives the best performance.

<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm 1.5.3
Did I forget something to enable this ?


Rob Verduijn
Christophe Fergeau
2016-09-14 07:53:26 UTC
Permalink
Post by Rob Verduijn
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm 1.5.3
Did I forget something to enable this ?
gl enable is not going to give you remote for now. The "Since 1.3.3" in
libvirt documentation refers to libvirt version, not QEMU. You need very
new libvirt/spice/qemu/mesa/... for things to work. I would not expect
these to be in centos.

Christophe
Rob Verduijn
2016-09-14 10:20:23 UTC
Permalink
Post by Christophe Fergeau
Post by Rob Verduijn
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm 1.5.3
Did I forget something to enable this ?
gl enable is not going to give you remote for now. The "Since 1.3.3" in
libvirt documentation refers to libvirt version, not QEMU. You need very
new libvirt/spice/qemu/mesa/... for things to work. I would not expect
these to be in centos.
Christophe
thanks for the update,

I checked the rhel7.3 beta and currently it has libvirt 2.0.0 so that one
will be in centos 7.3 as well.
So I have to wait for centos 7.3 which will be available shortly after
rhel7.3 is released (1 or 2 months ??)
What are the other version requirements ? (is there an url somewhere ?)
Is mesa also required on the kvm host or just the guests ?

Rob Verduijn
Christophe Fergeau
2016-09-14 10:28:18 UTC
Permalink
Post by Rob Verduijn
Post by Christophe Fergeau
Post by Rob Verduijn
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm 1.5.3
Did I forget something to enable this ?
gl enable is not going to give you remote for now. The "Since 1.3.3" in
libvirt documentation refers to libvirt version, not QEMU. You need very
new libvirt/spice/qemu/mesa/... for things to work. I would not expect
these to be in centos.
Christophe
thanks for the update,
I checked the rhel7.3 beta and currently it has libvirt 2.0.0 so that one
will be in centos 7.3 as well.
So I have to wait for centos 7.3 which will be available shortly after
rhel7.3 is released (1 or 2 months ??)
What are the other version requirements ? (is there an url somewhere ?)
Is mesa also required on the kvm host or just the guests ?
http://blog.wikichoon.com/2016/05/spice-openglvirgl-acceleration-on.html
has some data on the versions required. Gerd Hoffmann's slides from his
presentation at KVM Forum this year also had some version numbers if I'm
not mistaken. 7.3 is not going to have this support available/enabled.
And once again, this is local-only (emphasizing this as I got the
impression from the initial email that you were trying to improve remote
performance).

Christophe
Rob Verduijn
2016-09-14 10:37:02 UTC
Permalink
Ok local only as the article says.
Still nice to relieve my local vm's a bit.

Cheers
Rob Verduijn
Post by Christophe Fergeau
Post by Rob Verduijn
Post by Christophe Fergeau
Post by Rob Verduijn
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm
1.5.3
Post by Rob Verduijn
Post by Christophe Fergeau
Post by Rob Verduijn
Did I forget something to enable this ?
gl enable is not going to give you remote for now. The "Since 1.3.3" in
libvirt documentation refers to libvirt version, not QEMU. You need
very
Post by Rob Verduijn
Post by Christophe Fergeau
new libvirt/spice/qemu/mesa/... for things to work. I would not expect
these to be in centos.
Christophe
thanks for the update,
I checked the rhel7.3 beta and currently it has libvirt 2.0.0 so that one
will be in centos 7.3 as well.
So I have to wait for centos 7.3 which will be available shortly after
rhel7.3 is released (1 or 2 months ??)
What are the other version requirements ? (is there an url somewhere ?)
Is mesa also required on the kvm host or just the guests ?
http://blog.wikichoon.com/2016/05/spice-openglvirgl-acceleration-on.html
has some data on the versions required. Gerd Hoffmann's slides from his
presentation at KVM Forum this year also had some version numbers if I'm
not mistaken. 7.3 is not going to have this support available/enabled.
And once again, this is local-only (emphasizing this as I got the
impression from the initial email that you were trying to improve remote
performance).
Christophe
Rob Verduijn
2016-09-14 10:34:04 UTC
Permalink
Post by Rob Verduijn
Post by Christophe Fergeau
Post by Rob Verduijn
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm
1.5.3
Post by Rob Verduijn
Did I forget something to enable this ?
gl enable is not going to give you remote for now. The "Since 1.3.3" in
libvirt documentation refers to libvirt version, not QEMU. You need very
new libvirt/spice/qemu/mesa/... for things to work. I would not expect
these to be in centos.
Christophe
thanks for the update,
I checked the rhel7.3 beta and currently it has libvirt 2.0.0 so that one
will be in centos 7.3 as well.
So I have to wait for centos 7.3 which will be available shortly after
rhel7.3 is released (1 or 2 months ??)
What are the other version requirements ? (is there an url somewhere ?)
Is mesa also required on the kvm host or just the guests ?
Rob Verduijn1.5.3
Found the url :
https://www.spice-space.org/spice-user-manual.html
rhel7.3beta currently has
libvirt 2.0.0
mesa-libGL 11.2.2
mesa-libGLU 9.0.0
spice-server 0.12.4
qemu-kvm 1.5.3
qemu-guest-agent 2.5.0


If this is enough then centos 7.3 will be able to use gl.
Rob Verduijn
Christophe Fergeau
2016-09-14 10:39:48 UTC
Permalink
Post by Rob Verduijn
https://www.spice-space.org/spice-user-manual.html
rhel7.3beta currently has
libvirt 2.0.0
mesa-libGL 11.2.2
mesa-libGLU 9.0.0
spice-server 0.12.4
qemu-kvm 1.5.3
qemu-guest-agent 2.5.0
If this is enough then centos 7.3 will be able to use gl.
At least spice-server and qemu are too old.

Christophe
Frediano Ziglio
2016-09-14 10:59:42 UTC
Permalink
Post by Rob Verduijn
Post by Christophe Fergeau
Hey,
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
I would try to minimize latency, and to get as much bandwidth as
possible, probably some tweaks can be done for that on the openvpn side?
Regarding QEMU, make sure you are using the QXL video driver as well as
the SPICE agent in the guest. There are also a bunch of compression
options that you can try to tweak, see
https://libvirt.org/formatdomain.html#elementsGraphics
Christophe
bandwith and latency are not mine to decide about
You could tweak OpenVPN settings too.
Post by Rob Verduijn
QXL-driver check
spice-agent check
<graphics type='spice' port='5999' autoport='no' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
<image compression='glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>
</graphics>
I tried them on auto_glz,auto,auto,filter (tried every combo)
But I guess my bandwith is not stable and since setting them like this gives
the best performance.
<gl enable='yes'/> gets an error if I add that to the config.
It says this option is included since 1.3.3 and centos has qemu-kvm 1.5.3
Did I forget something to enable this ?
Rob Verduijn
Frediano
Frediano Ziglio
2016-09-14 13:49:43 UTC
Permalink
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and delays
when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?

The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e699f0a07a795.

Be prepare for the rollback, it's not a version meant for production usage.

Frediano
Rob Verduijn
2016-09-14 13:54:12 UTC
Permalink
thanx,I'll stick with the centos packages,

I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.

Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me) because
the firewall is managed by a third party.

Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Frediano Ziglio
2016-09-14 13:59:07 UTC
Permalink
Could you test at least? Would be very helpful. We could then backport some improvements.

Frediano
Post by Rob Verduijn
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are related
to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me) because
the firewall is managed by a third party.
Rob Verduijn
Post by Frediano Ziglio
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and delays
when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is
0e2191c363e109475aeb2bff401e699f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Rob Verduijn
2016-09-15 05:42:11 UTC
Permalink
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?

Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport some improvements.
Frediano
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me) because
the firewall is managed by a third party.
Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Frediano Ziglio
2016-09-15 11:31:11 UTC
Permalink
I saw you are using CentOS 7. I built the package with RHEL 7 (they are binary compatible).
About the testing just which normal usage you should see improvements in bandwidth and
reactivity.

Changes from current CentOS package:
- used a newer version, there are couple of changes that decrease latency;
- additional patches to improve bandwidth usage (for small drawing this should decrease
bandwidth usage by a 15-20%);
- additional patch to decrease a bandwidth limitation due to a peculiar half-duplex usage
of spice protocol (this is clearly visible with high latency connections);
- additional patch to decrease packet fragmentation due to TCP_NODELAY usage.

Alternatively would be helpful for us to get a local reproduction of the problem.
OpenVPN configuration files would be helpful (we don't need any security
detail like keys, ip, host or system names, just to understand the type of VPN,
encryption parameters, compression, additional latency introduced and so on).

The fact that you are not able to get a record from the guest means that the QXL
(guest <-> server) protocol how the spice-server is handling guest command is
fine. The fact that on the client you can see clearly such slowness is due to spice
protocol, the connection/vpn, some spice-server implementation and possibly
client implementation too. Unfortunately too much stuff to be able to point the
finger to one of them.
I tried some test and did this:
- opened task manager on Windows 7;
- switched to performance tab;
- maximized task manager;
- double clicked on CPU usage to get only CPU usage and history.
When CPU usage change I can see the flickering on CPU usage but not on
the history graphs. It this the kind of flickering you are noticing?

Frediano
Post by Rob Verduijn
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?
Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport some
improvements.
Frediano
Post by Rob Verduijn
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are related
to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me) because
the firewall is managed by a third party.
Rob Verduijn
Post by Frediano Ziglio
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays
when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is
0e2191c363e109475aeb2bff401e699f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Rob Verduijn
2016-09-15 14:04:51 UTC
Permalink
Hello,

The performance fixes sound awesome.
I'm afraid I cannot test them in the environment with the low bandwith
setup soon (maybe next week, but don't hold your breath)
I'm going to build a local test setup to see if this is improving some of
the issues. I got a laptop running the rhel7.3 beta, let's see how that
performs.

Rob Verduijn
Post by Frediano Ziglio
I saw you are using CentOS 7. I built the package with RHEL 7 (they are binary compatible).
About the testing just which normal usage you should see improvements in bandwidth and
reactivity.
- used a newer version, there are couple of changes that decrease latency;
- additional patches to improve bandwidth usage (for small drawing this should decrease
bandwidth usage by a 15-20%);
- additional patch to decrease a bandwidth limitation due to a peculiar half-duplex usage
of spice protocol (this is clearly visible with high latency
connections);
- additional patch to decrease packet fragmentation due to TCP_NODELAY usage.
Alternatively would be helpful for us to get a local reproduction of the problem.
OpenVPN configuration files would be helpful (we don't need any security
detail like keys, ip, host or system names, just to understand the type of VPN,
encryption parameters, compression, additional latency introduced and so on).
The fact that you are not able to get a record from the guest means that the QXL
(guest <-> server) protocol how the spice-server is handling guest command is
fine. The fact that on the client you can see clearly such slowness is due to spice
protocol, the connection/vpn, some spice-server implementation and possibly
client implementation too. Unfortunately too much stuff to be able to point the
finger to one of them.
- opened task manager on Windows 7;
- switched to performance tab;
- maximized task manager;
- double clicked on CPU usage to get only CPU usage and history.
When CPU usage change I can see the flickering on CPU usage but not on
the history graphs. It this the kind of flickering you are noticing?
Frediano
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?
Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport some improvements.
Frediano
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me)
because the firewall is managed by a third party.
Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Rob Verduijn
2016-09-15 14:54:21 UTC
Permalink
Hi,

I tried a virtual rhel7.3beta (server with gui) on a rhel7.3beta host.

The host was a laptop to which I setup to use my old wifi router that only
has 54Mbit so the bandwith was poor and unstable.

Without the compression options the spice display was really bad.
You could see the screen being rendered bit by bit.

I tried this with the custom rpm and did not really see an improvement.

I downgraded the spice-server and rebooted

I added the compression options:
<image compression='glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>

Now the spice display was very usable and almost no artifacts.

Only when moving the windows you can see tearing at the edges.
Also started a glxgears window to see how it deals with that

I applied the custom rpm and rebooted the host.

After restarting the vm and logging into spice
There is less tearing when moving the window.

So there is an improvement, but it's hard to tell how much by simply
watching when draging the window.


some details from our openvpn config
the protocol has to be tcp.....

centos 7.2 openvpn server settings:

port secret-port-number
proto tcp
dev tun
tun-mtu 1500
ca /path/to/ca
cert /path/to/cert
key /path/to/key
dh /path/to/dh
server some.ip.address some.subnet.mask
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status /path/to/log
verb 3
mute 20

fedora 24 NetworkManager settings:
[connection]
id=our id for our top secret vpn
uuid=xxxxxxxx.xxxx.xxx.xxx.xx
type=vpn
permissions=user:secretusernottellingyou:;
secondaries=

[vpn]
remote-random=no
connection-type=tls
remote=remote.ip.address
tunnel-mtu=1500
comp-lzo=yes
cert-pass-flags=1
proto-tcp=yes
port=<secret-port>
mssfix=yes
dev-type=tun
ca=/path/to/ca
key=/path/to/key
cert=/path/to/crt
service-type=org.freedesktop.NetworkManager.openvpn

[ipv4]
dns-search=
ignore-auto-routes=true
method=auto
never-default=true

route1=additional.route/mask

route2=additional.route/mask



[ipv6]

addr-gen-mode=stable-privacy

dns-search=

method=auto



Cheers
Rob Verduijn.
Post by Rob Verduijn
Hello,
The performance fixes sound awesome.
I'm afraid I cannot test them in the environment with the low bandwith
setup soon (maybe next week, but don't hold your breath)
I'm going to build a local test setup to see if this is improving some of
the issues. I got a laptop running the rhel7.3 beta, let's see how that
performs.
Rob Verduijn
Post by Frediano Ziglio
I saw you are using CentOS 7. I built the package with RHEL 7 (they are
binary compatible).
About the testing just which normal usage you should see improvements in bandwidth and
reactivity.
- used a newer version, there are couple of changes that decrease latency;
- additional patches to improve bandwidth usage (for small drawing this should decrease
bandwidth usage by a 15-20%);
- additional patch to decrease a bandwidth limitation due to a peculiar half-duplex usage
of spice protocol (this is clearly visible with high latency connections);
- additional patch to decrease packet fragmentation due to TCP_NODELAY usage.
Alternatively would be helpful for us to get a local reproduction of the problem.
OpenVPN configuration files would be helpful (we don't need any security
detail like keys, ip, host or system names, just to understand the type of VPN,
encryption parameters, compression, additional latency introduced and so on).
The fact that you are not able to get a record from the guest means that the QXL
(guest <-> server) protocol how the spice-server is handling guest command is
fine. The fact that on the client you can see clearly such slowness is due to spice
protocol, the connection/vpn, some spice-server implementation and possibly
client implementation too. Unfortunately too much stuff to be able to point the
finger to one of them.
- opened task manager on Windows 7;
- switched to performance tab;
- maximized task manager;
- double clicked on CPU usage to get only CPU usage and history.
When CPU usage change I can see the flickering on CPU usage but not on
the history graphs. It this the kind of flickering you are noticing?
Frediano
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?
Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport
some improvements.
Frediano
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me)
because the firewall is managed by a third party.
Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Frediano Ziglio
2016-09-29 11:21:42 UTC
Permalink
Post by Rob Verduijn
Hi,
I tried a virtual rhel7.3beta (server with gui) on a rhel7.3beta host.
The host was a laptop to which I setup to use my old wifi router that only
has 54Mbit so the bandwith was poor and unstable.
Without the compression options the spice display was really bad.
You could see the screen being rendered bit by bit.
I tried this with the custom rpm and did not really see an improvement.
I downgraded the spice-server and rebooted
<image compression='glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>
Now the spice display was very usable and almost no artifacts.
Only when moving the windows you can see tearing at the edges.
Also started a glxgears window to see how it deals with that
I applied the custom rpm and rebooted the host.
After restarting the vm and logging into spice
There is less tearing when moving the window.
Good! Thanks for the test! I'll work to make these changes integrated in a future version.
Post by Rob Verduijn
So there is an improvement, but it's hard to tell how much by simply watching
when draging the window.
some details from our openvpn config
the protocol has to be tcp.....
port secret-port-number
proto tcp
dev tun
tun-mtu 1500
ca /path/to/ca
cert /path/to/cert
key /path/to/key
dh /path/to/dh
server some.ip.address some.subnet.mask
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status /path/to/log
verb 3
mute 20
I would personally try different combination of compression/encryption.
Obviously you want encryption on the VPN but if you have also encryption on Spice this
is mainly wasting bandwidth and latency.
Also depending on the type of data VPN compression can be a bad idea.
I would try (if this is possible) to turn it off.

On the Windows side I tried to see what could cause the flicker issue.
It seems for some reason (unknown to me) some commands are trying to read some data back
from the card causing a 0.01 seconds delay (which could be one cause of the delay). No much
to suggest to improve this. Looks like the XPDM driver can be compiled using different way
to do the synchronization and looks like the new WDDM driver does the synchronization in
a different way (not clear why).
I don't think I'll have much more time to investigate into this.
I would suggest to open a ticket upstream so people don't forget this.

Frediano
Post by Rob Verduijn
[connection]
id=our id for our top secret vpn
uuid=xxxxxxxx.xxxx.xxx.xxx.xx
type=vpn
permissions=user:secretusernottellingyou:;
secondaries=
[vpn]
remote-random=no
connection-type=tls
remote=remote.ip.address
tunnel-mtu=1500
comp-lzo=yes
cert-pass-flags=1
proto-tcp=yes
port=<secret-port>
mssfix=yes
dev-type=tun
ca=/path/to/ca
key=/path/to/key
cert=/path/to/crt
service-type=org.freedesktop.NetworkManager.openvpn
[ipv4]
dns-search=
ignore-auto-routes=true
method=auto
never-default=true
route1=additional.route/mask
route2=additional.route/mask
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto
Cheers
Rob Verduijn.
Post by Rob Verduijn
Hello,
The performance fixes sound awesome.
I'm afraid I cannot test them in the environment with the low bandwith
setup
soon (maybe next week, but don't hold your breath)
I'm going to build a local test setup to see if this is improving some of
the
issues. I got a laptop running the rhel7.3 beta, let's see how that
performs.
Rob Verduijn
Post by Frediano Ziglio
I saw you are using CentOS 7. I built the package with RHEL 7 (they are
binary compatible).
About the testing just which normal usage you should see improvements in
bandwidth and
reactivity.
- used a newer version, there are couple of changes that decrease latency;
- additional patches to improve bandwidth usage (for small drawing this
should decrease
bandwidth usage by a 15-20%);
- additional patch to decrease a bandwidth limitation due to a peculiar
half-duplex usage
of spice protocol (this is clearly visible with high latency connections);
- additional patch to decrease packet fragmentation due to TCP_NODELAY usage.
Alternatively would be helpful for us to get a local reproduction of the problem.
OpenVPN configuration files would be helpful (we don't need any security
detail like keys, ip, host or system names, just to understand the type
of
VPN,
encryption parameters, compression, additional latency introduced and so on).
The fact that you are not able to get a record from the guest means that
the
QXL
(guest <-> server) protocol how the spice-server is handling guest
command
is
fine. The fact that on the client you can see clearly such slowness is
due
to
spice
protocol, the connection/vpn, some spice-server implementation and possibly
client implementation too. Unfortunately too much stuff to be able to
point
the
finger to one of them.
- opened task manager on Windows 7;
- switched to performance tab;
- maximized task manager;
- double clicked on CPU usage to get only CPU usage and history.
When CPU usage change I can see the flickering on CPU usage but not on
the history graphs. It this the kind of flickering you are noticing?
Frediano
Post by Rob Verduijn
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?
Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport
some
improvements.
Frediano
Post by Rob Verduijn
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related
to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me)
because
the firewall is managed by a third party.
Rob Verduijn
Post by Frediano Ziglio
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build
up,
and
delays
when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd
party.
I have full access to the kvm host, kvm guest and openvpn
server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for
guests
and
host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is
0e2191c363e109475aeb2bff401e699f0a07a795.
Be prepare for the rollback, it's not a version meant for
production
usage.
Frediano
Rob Verduijn
2016-09-29 13:04:45 UTC
Permalink
Hi,

I'm not using TLS to connect the spice session, so unless there is another
encryption that I'm not aware of, it's already off
And I already tried switching of lzo compression, but if there was an
improvement I couldn't see it.

I'm not sure where the microsoft windows display driver entered the
discussion,
whenever I said window I was referring to one of the square boxes on my
linux desktop that contains the application that I'm lookinf at
(firefox/terminal/chrome).

Rob Verduijn
Post by Rob Verduijn
Hi,
I tried a virtual rhel7.3beta (server with gui) on a rhel7.3beta host.
The host was a laptop to which I setup to use my old wifi router that only
has 54Mbit so the bandwith was poor and unstable.
Without the compression options the spice display was really bad.
You could see the screen being rendered bit by bit.
I tried this with the custom rpm and did not really see an improvement.
I downgraded the spice-server and rebooted
<image compression='glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>
Now the spice display was very usable and almost no artifacts.
Only when moving the windows you can see tearing at the edges.
Also started a glxgears window to see how it deals with that
I applied the custom rpm and rebooted the host.
After restarting the vm and logging into spice
There is less tearing when moving the window.
Good! Thanks for the test! I'll work to make these changes integrated in a future version.
So there is an improvement, but it's hard to tell how much by simply
watching when draging the window.
some details from our openvpn config
the protocol has to be tcp.....
port secret-port-number
proto tcp
dev tun
tun-mtu 1500
ca /path/to/ca
cert /path/to/cert
key /path/to/key
dh /path/to/dh
server some.ip.address some.subnet.mask
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status /path/to/log
verb 3
mute 20
I would personally try different combination of compression/encryption.
Obviously you want encryption on the VPN but if you have also encryption on Spice this
is mainly wasting bandwidth and latency.
Also depending on the type of data VPN compression can be a bad idea.
I would try (if this is possible) to turn it off.
On the Windows side I tried to see what could cause the flicker issue.
It seems for some reason (unknown to me) some commands are trying to read some data back
from the card causing a 0.01 seconds delay (which could be one cause of the delay). No much
to suggest to improve this. Looks like the XPDM driver can be compiled using different way
to do the synchronization and looks like the new WDDM driver does the synchronization in
a different way (not clear why).
I don't think I'll have much more time to investigate into this.
I would suggest to open a ticket upstream so people don't forget this.
Frediano
[connection]
id=our id for our top secret vpn
uuid=xxxxxxxx.xxxx.xxx.xxx.xx
type=vpn
permissions=user:secretusernottellingyou:;
secondaries=
[vpn]
remote-random=no
connection-type=tls
remote=remote.ip.address
tunnel-mtu=1500
comp-lzo=yes
cert-pass-flags=1
proto-tcp=yes
port=<secret-port>
mssfix=yes
dev-type=tun
ca=/path/to/ca
key=/path/to/key
cert=/path/to/crt
service-type=org.freedesktop.NetworkManager.openvpn
[ipv4]
dns-search=
ignore-auto-routes=true
method=auto
never-default=true
route1=additional.route/mask
route2=additional.route/mask
[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto
Cheers
Rob Verduijn.
Post by Rob Verduijn
Hello,
The performance fixes sound awesome.
I'm afraid I cannot test them in the environment with the low bandwith
setup soon (maybe next week, but don't hold your breath)
I'm going to build a local test setup to see if this is improving some of
the issues. I got a laptop running the rhel7.3 beta, let's see how that
performs.
Rob Verduijn
Post by Frediano Ziglio
I saw you are using CentOS 7. I built the package with RHEL 7 (they are
binary compatible).
About the testing just which normal usage you should see improvements in bandwidth and
reactivity.
- used a newer version, there are couple of changes that decrease latency;
- additional patches to improve bandwidth usage (for small drawing this should decrease
bandwidth usage by a 15-20%);
- additional patch to decrease a bandwidth limitation due to a peculiar
half-duplex usage
of spice protocol (this is clearly visible with high latency connections);
- additional patch to decrease packet fragmentation due to TCP_NODELAY usage.
Alternatively would be helpful for us to get a local reproduction of the problem.
OpenVPN configuration files would be helpful (we don't need any security
detail like keys, ip, host or system names, just to understand the type of VPN,
encryption parameters, compression, additional latency introduced and so on).
The fact that you are not able to get a record from the guest means that the QXL
(guest <-> server) protocol how the spice-server is handling guest command is
fine. The fact that on the client you can see clearly such slowness is due to spice
protocol, the connection/vpn, some spice-server implementation and possibly
client implementation too. Unfortunately too much stuff to be able to point the
finger to one of them.
- opened task manager on Windows 7;
- switched to performance tab;
- maximized task manager;
- double clicked on CPU usage to get only CPU usage and history.
When CPU usage change I can see the flickering on CPU usage but not on
the history graphs. It this the kind of flickering you are noticing?
Frediano
For which distro is that package ?
Centos 7.2 ? rhel7.3beta or fedora24 ?
Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport
some improvements.
Frediano
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me)
because the firewall is managed by a third party.
Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Rob Verduijn
2016-09-15 05:43:23 UTC
Permalink
Or rather,

Could you specify what you would like to see tested ?
Setup of the host / guest ?
specific guest definitions ?

Rob Verduijn
Post by Frediano Ziglio
Could you test at least? Would be very helpful. We could then backport some improvements.
Frediano
thanx,I'll stick with the centos packages,
I need a very good reason before I start using beta packages.
And a nice to have feature is not one of them.
Also I dug in to the openvpn tweaks and it seems that all of them are
related to udp tunnels.
Performance is sadly rather low when you have to use tcp (like me) because
the firewall is managed by a third party.
Rob Verduijn
Post by Rob Verduijn
Hello,
I'm trying to improve my spice performance on a kvm host/guest.
It's currently rather slow and I can see screens beeing build up, and
delays when draging windows.
It's being tunneled through openvpn, which is set to use tcp.
tcp required because of the firewall which is maintained by 3rd party.
I have full access to the kvm host, kvm guest and openvpn server.
Have you got any tips so that I can improve spice performance ?
I alrready am running tuned with the virtual-guest profile for guests
and host profile for the host.
All systems are runnning CentOS 7
- the KVM host ?
- the KVM guest ?
- the openvpn server ?
Cheers
Rob Verduijn
Hi,
can you try version at https://www.datafilehost.com/d/b07f008e ?
The sha1 hash (please check it) is 0e2191c363e109475aeb2bff401e69
9f0a07a795.
Be prepare for the rollback, it's not a version meant for production usage.
Frediano
Continue reading on narkive:
Loading...