<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 27, 2016 at 7:30 PM, Konrad Rzeszutek Wilk <span dir="ltr"><<a href="mailto:konrad.wilk@oracle.com" target="_blank">konrad.wilk@oracle.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">On Sat, Jan 23, 2016 at 05:12:04PM +0100, Tommi Airikka wrote:<br>
> Xen developers,<br>
><br>
> After an upgrade of my Debian Jessie dom0 and domUs, my passthroughed<br>
> NIC stopped working.<br>
> This bug was probably introduced in Debian Jessie sometime<br>
> between 2015-12-30 and 2016-01-08 as 2015-12-30 as 2015-12-30 was the<br>
> last time I upgraded without any problems according to my dpkg.log.<br>
<br>
</span>This upgrade looks to only have upgraded the hypervisor?<br>
<br>
As in I see:<br>
<span class=""><br>
domU "bug" "uname -a":<br>
Linux bug 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u2 (2016-01-02)<br>
x86_64 GNU/Linux<br>
<br>
domU "working" "uname -a":<br>
Linux working 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u2<br>
(2016-01-02) x86_64 GNU/Linux<br>
<br>
</span>So same version? What was the earlier version of the hypervisor?<br></blockquote><div><br></div><div>I forgot to mention that I use apt-dater to upgrade both dom0 and domUs at the same time. domU "bug" was created for debugging purposes just a couple of hours after I used apt-dater and realized that there's something wrong with the pci passthrough. domU "bug" and domU "working" should have relatively similar software state.<br><br></div><div>According to dom0 dpkg.log, the xen-hypervisor-4.4-amd64 (version "4.4.1-9+deb8u3") has been untouched since 2015-12-21.<br>linux-image-3.16.0-4-amd64 was upgraded from "3.16.7-ckt20-1+deb8u1" to "3.16.7-ckt20-1+deb8u2" at the point of time when pci passthrough stopped working.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
The Xen version you have says:<br>
> (XEN) I/O virtualisation disabled<br>
<br>
Which is not very healthy for PCI passthrough. Albeit you can do<br>
it with PV without IOMMU. Did the previous version of Xen have the same<br>
message?<br></blockquote><div><br></div><div>I downgraded linux-image on dom0:<br>dpkg -i linux-image-3.16.0-4-amd64_3.16.7-ckt20-1+deb8u1_amd64.deb<br><br></div><div>and now the pci passthrough seems to work!<br></div><div><br>dom0 'uname -a':<br>Linux dom0 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u1 (2015-12-14) x86_64 GNU/Linux<br></div><div><br></div><div>domU bug 'uname -a':<br>Linux bug 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u2 (2016-01-02) x86_64 GNU/Linux<br><br></div><div>But I still have the "(XEN) I/O virtualisation disabled".<br></div><div>dom0 'xl dmesg':<br><br>(XEN) Xen version 4.4.1 (Debian 4.4.1-9+deb8u3) (<a href="mailto:ultrotter@debian.org">ultrotter@debian.org</a>) (gcc (Debian 4.9.2-10) 4.9.2) debug=n Wed Nov 25 15:46:22 EET 2015<br>(XEN) Bootloader: GRUB 2.02~beta2-22+deb8u1<br>(XEN) Command line: placeholder dom0_mem=1024M,max:1024M<br>(XEN) Video information:<br>(XEN) VGA is text mode 80x25, font 8x16<br>(XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds<br>(XEN) EDID info not retrieved because no DDC retrieval method detected<br>(XEN) Disc information:<br>(XEN) Found 4 MBR signatures<br>(XEN) Found 2 EDD information structures<br>(XEN) Xen-e820 RAM map:<br>(XEN) 0000000000000000 - 0000000000099800 (usable)<br>(XEN) 0000000000099800 - 0000000000099c00 (reserved)<br>(XEN) 000000000009e000 - 00000000000a0000 (reserved)<br>(XEN) 00000000000f0000 - 0000000000100000 (reserved)<br>(XEN) 0000000000100000 - 00000000f1de4000 (usable)<br>(XEN) 00000000f1de4000 - 00000000f1dee000 (ACPI data)<br>(XEN) 00000000f1dee000 - 00000000f8000000 (reserved)<br>(XEN) 00000000fec00000 - 00000000fee10000 (reserved)<br>(XEN) 00000000ff800000 - 0000000100000000 (reserved)<br>(XEN) 0000000100000000 - 000000028bfff000 (usable)<br>(XEN) ACPI: RSDP 000F4F00, 0024 (r2 HP )<br>(XEN) ACPI: XSDT F1DE6400, 00B4 (r1 HP ProLiant 2 � 162E)<br>(XEN) ACPI: FACP F1DE6540, 00F4 (r3 HP ProLiant 2 � 162E)<br>(XEN) ACPI: DSDT F1DE6640, 1C1A (r1 HP DSDT 1 INTL 20030228)<br>(XEN) ACPI: FACS F1DE4140, 0040<br>(XEN) ACPI: SPCR F1DE4180, 0050 (r1 HP SPCRRBSU 1 � 162E)<br>(XEN) ACPI: MCFG F1DE4200, 003C (r1 HP ProLiant 1 0)<br>(XEN) ACPI: HPET F1DE4240, 0038 (r1 HP ProLiant 2 � 162E)<br>(XEN) ACPI: FFFF F1DE4280, 0064 (r2 HP ProLiant 2 � 162E)<br>(XEN) ACPI: SPMI F1DE4300, 0040 (r5 HP ProLiant 1 � 162E)<br>(XEN) ACPI: ERST F1DE4340, 0230 (r1 HP ProLiant 1 � 162E)<br>(XEN) ACPI: APIC F1DE4580, 0252 (r1 HP ProLiant 2 0)<br>(XEN) ACPI: FFFF F1DE4800, 0176 (r1 HP ProLiant 1 � 162E)<br>(XEN) ACPI: BERT F1DE4980, 0030 (r1 HP ProLiant 1 � 162E)<br>(XEN) ACPI: HEST F1DE49C0, 00BC (r1 HP ProLiant 1 � 162E)<br>(XEN) ACPI: FFFF F1DE4A80, 1914 (r1 HP ProLiant 1 � 162E)<br>(XEN) ACPI: FFFF F1DE63C0, 002D (r1 HP ProLiant 1 0)<br>(XEN) ACPI: SSDT F1DE8280, 0137 (r3 HP CRSPCI0 2 HP 1)<br>(XEN) ACPI: SSDT F1DE83C0, 0177 (r3 HP riser0 2 INTL 20030228)<br>(XEN) ACPI: SSDT F1DE8540, 01E1 (r1 HP pcc 1 INTL 20090625)<br>(XEN) ACPI: SSDT F1DE8740, 0377 (r1 HP pmab 1 INTL 20090625)<br>(XEN) ACPI: SSDT F1DE8AC0, 0504 (r1 INTEL PPM RCM 80000001 INTL 20061109)<br>(XEN) System RAM: 10205MB (10450416kB)<br>(XEN) Domain heap initialised<br>(XEN) Processor #0 7:10 APIC version 21<br>(XEN) Processor #2 7:10 APIC version 21<br>(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23<br>(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs<br>(XEN) Using scheduler: SMP Credit Scheduler (credit)<br>(XEN) Detected 2294.859 MHz processor.<br>(XEN) Initing memory sharing.<br>(XEN) xstate_init: using cntxt_size: 0x240 and states: 0x3<br>(XEN) I/O virtualisation disabled<br>(XEN) Enabled directed EOI with ioapic_ack_old on!<br>(XEN) ENABLING IO-APIC IRQs<br>(XEN) -> Using old ACK method<br>(XEN) Platform timer is 14.318MHz HPET<br>(XEN) Allocated console ring of 16 KiB.<br>(XEN) VMX: Supported advanced features:<br>(XEN) - APIC MMIO access virtualisation<br>(XEN) - APIC TPR shadow<br>(XEN) - Extended Page Tables (EPT)<br>(XEN) - Virtual-Processor Identifiers (VPID)<br>(XEN) - Virtual NMI<br>(XEN) - MSR direct-access bitmap<br>(XEN) - Unrestricted Guest<br>(XEN) HVM: ASIDs enabled.<br>(XEN) HVM: VMX enabled<br>(XEN) HVM: Hardware Assisted Paging (HAP) detected<br>(XEN) HVM: HAP page sizes: 4kB, 2MB<br>(XEN) Brought up 2 CPUs<br>(XEN) *** LOADING DOMAIN 0 ***<br>(XEN) Xen kernel: 64-bit, lsb, compat32<br>(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1f18000<br>(XEN) PHYSICAL MEMORY ARRANGEMENT:<br>(XEN) Dom0 alloc.: 000000023c000000->0000000240000000 (241836 pages to be allocated)<br>(XEN) Init. ramdisk: 000000028aeac000->000000028bdff9af<br>(XEN) VIRTUAL MEMORY ARRANGEMENT:<br>(XEN) Loaded kernel: ffffffff81000000->ffffffff81f18000<br>(XEN) Init. ramdisk: ffffffff81f18000->ffffffff82e6b9af<br>(XEN) Phys-Mach map: ffffffff82e6c000->ffffffff8306c000<br>(XEN) Start info: ffffffff8306c000->ffffffff8306c4b4<br>(XEN) Page tables: ffffffff8306d000->ffffffff8308a000<br>(XEN) Boot stack: ffffffff8308a000->ffffffff8308b000<br>(XEN) TOTAL: ffffffff80000000->ffffffff83400000<br>(XEN) ENTRY ADDRESS: ffffffff819021f0<br>(XEN) Dom0 has maximum 2 VCPUs<br>(XEN) Scrubbing Free RAM: ..........................................................................................done.<br>(XEN) Initial low memory virq threshold set at 0x4000 pages.<br>(XEN) Std. Loglevel: Errors and warnings<br>(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)<br>(XEN) Xen is relinquishing VGA console.<br>(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)<br>(XEN) Freed 284kB init memory.<br>(XEN) mm.c:812: d13: Forcing read-only access to MFN fbff1<br>(XEN) mm.c:812: d13: Forcing read-only access to MFN fbff1<br>(XEN) mm.c:812: d13: Forcing read-only access to MFN fbff1<br>(XEN) mm.c:812: d0: Forcing read-only access to MFN fbff1<br>(XEN) mm.c:812: d13: Forcing read-only access to MFN faba0<br>(XEN) mm.c:812: d0: Forcing read-only access to MFN faba0<br></div><div><br></div><div>domU "bug" 'dmesg':<br><br>[ 0.000000] Initializing cgroup subsys cpuset<br>[ 0.000000] Initializing cgroup subsys cpu<br>[ 0.000000] Initializing cgroup subsys cpuacct<br>[ 0.000000] Linux version 3.16.0-4-amd64 (<a href="mailto:debian-kernel@lists.debian.org">debian-kernel@lists.debian.org</a>) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt20-1+deb8u2 (2016-01-02)<br>[ 0.000000] Command line: root=/dev/xvda2 ro elevator=noop root=/dev/xvda2 ro <br>[ 0.000000] ACPI in unprivileged domain disabled<br>[ 0.000000] 1-1 mapping on 8000->100000<br>[ 0.000000] Set 1015808 page(s) to 1-1 mapping<br>[ 0.000000] 1-1 mapping on 100000->8000000<br>[ 0.000000] e820: BIOS-provided physical RAM map:<br>[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable<br>[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved<br>[ 0.000000] Xen: [mem 0x0000000000100000-0x0000000007ffffff] usable<br>[ 0.000000] Xen: [mem 0x0000000008000000-0x00000000f1de3fff] unusable<br>[ 0.000000] Xen: [mem 0x00000000f1de4000-0x00000000f1dedfff] ACPI data<br>[ 0.000000] Xen: [mem 0x00000000f1dee000-0x00000000f7ffffff] reserved<br>[ 0.000000] Xen: [mem 0x00000000fec00000-0x00000000feefffff] reserved<br>[ 0.000000] Xen: [mem 0x00000000ff800000-0x00000000ffffffff] reserved<br>[ 0.000000] NX (Execute Disable) protection: active<br>[ 0.000000] DMI not present or invalid.<br>[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved<br>[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable<br>[ 0.000000] AGP: No AGP bridge found<br>[ 0.000000] e820: last_pfn = 0x8000 max_arch_pfn = 0x400000000<br>[ 0.000000] Base memory trampoline at [ffff88000009a000] 9a000 size 24576<br>[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]<br>[ 0.000000] [mem 0x00000000-0x000fffff] page 4k<br>[ 0.000000] init_memory_mapping: [mem 0x07e00000-0x07ffffff]<br>[ 0.000000] [mem 0x07e00000-0x07ffffff] page 4k<br>[ 0.000000] BRK [0x01b02000, 0x01b02fff] PGTABLE<br>[ 0.000000] init_memory_mapping: [mem 0x04000000-0x07dfffff]<br>[ 0.000000] [mem 0x04000000-0x07dfffff] page 4k<br>[ 0.000000] BRK [0x01b03000, 0x01b03fff] PGTABLE<br>[ 0.000000] BRK [0x01b04000, 0x01b04fff] PGTABLE<br>[ 0.000000] BRK [0x01b05000, 0x01b05fff] PGTABLE<br>[ 0.000000] BRK [0x01b06000, 0x01b06fff] PGTABLE<br>[ 0.000000] BRK [0x01b07000, 0x01b07fff] PGTABLE<br>[ 0.000000] init_memory_mapping: [mem 0x00100000-0x03ffffff]<br>[ 0.000000] [mem 0x00100000-0x03ffffff] page 4k<br>[ 0.000000] RAMDISK: [mem 0x01f18000-0x049bffff]<br>[ 0.000000] NUMA turned off<br>[ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000007ffffff]<br>[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x07ffffff]<br>[ 0.000000] NODE_DATA [mem 0x07fe7000-0x07febfff]<br>[ 0.000000] Zone ranges:<br>[ 0.000000] DMA [mem 0x00001000-0x00ffffff]<br>[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]<br>[ 0.000000] Normal empty<br>[ 0.000000] Movable zone start for each node<br>[ 0.000000] Early memory node ranges<br>[ 0.000000] node 0: [mem 0x00001000-0x0009ffff]<br>[ 0.000000] node 0: [mem 0x00100000-0x07ffffff]<br>[ 0.000000] On node 0 totalpages: 32671<br>[ 0.000000] DMA zone: 56 pages used for memmap<br>[ 0.000000] DMA zone: 21 pages reserved<br>[ 0.000000] DMA zone: 3999 pages, LIFO batch:0<br>[ 0.000000] DMA32 zone: 392 pages used for memmap<br>[ 0.000000] DMA32 zone: 28672 pages, LIFO batch:7<br>[ 0.000000] SFI: Simple Firmware Interface v0.81 <a href="http://simplefirmware.org">http://simplefirmware.org</a><br>[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs<br>[ 0.000000] nr_irqs_gsi: 16<br>[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]<br>[ 0.000000] e820: [mem 0xf8000000-0xfebfffff] available for PCI devices<br>[ 0.000000] Booting paravirtualized kernel on Xen<br>[ 0.000000] Xen version: 4.4.1 (preserve-AD)<br>[ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1<br>[ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff880007c00000 s80896 r8192 d21504 u2097152<br>[ 0.000000] pcpu-alloc: s80896 r8192 d21504 u2097152 alloc=1*2097152<br>[ 0.000000] pcpu-alloc: [0] 0 <br>[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 32202<br>[ 0.000000] Policy zone: DMA32<br>[ 0.000000] Kernel command line: root=/dev/xvda2 ro elevator=noop root=/dev/xvda2 ro <br>[ 0.000000] PID hash table entries: 512 (order: 0, 4096 bytes)<br>[ 0.000000] xsave: enabled xstate_bv 0x3, cntxt size 0x240<br>[ 0.000000] AGP: Checking aperture...<br>[ 0.000000] AGP: No AGP bridge found<br>[ 0.000000] Calgary: detecting Calgary via BIOS EBDA area<br>[ 0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!<br>[ 0.000000] Memory: 72968K/130684K available (5219K kernel code, 947K rwdata, 1836K rodata, 1204K init, 840K bss, 57716K reserved)<br>[ 0.000000] Hierarchical RCU implementation.<br>[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.<br>[ 0.000000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.<br>[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1<br>[ 0.000000] NR_IRQS:33024 nr_irqs:256 16<br>[ 0.000000] xen:events: Using FIFO-based ABI<br>[ 0.000000] Console: colour dummy device 80x25<br>[ 0.000000] console [tty0] enabled<br>[ 0.000000] console [hvc0] enabled<br>[ 0.000000] Xen: using vcpuop timer interface<br>[ 0.000000] installing Xen timer for CPU 0<br>[ 0.000000] tsc: Detected 2294.858 MHz processor<br>[ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4589.71 BogoMIPS (lpj=9179432)<br>[ 0.004000] pid_max: default: 32768 minimum: 301<br>[ 0.004000] Security Framework initialized<br>[ 0.004000] AppArmor: AppArmor disabled by boot time parameter<br>[ 0.004000] Yama: disabled by default; enable with sysctl kernel.yama.*<br>[ 0.004000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)<br>[ 0.004000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)<br>[ 0.004000] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)<br>[ 0.004000] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)<br>[ 0.004000] Initializing cgroup subsys memory<br>[ 0.004000] Initializing cgroup subsys devices<br>[ 0.004000] Initializing cgroup subsys freezer<br>[ 0.004000] Initializing cgroup subsys net_cls<br>[ 0.004000] Initializing cgroup subsys blkio<br>[ 0.004000] Initializing cgroup subsys perf_event<br>[ 0.004000] Initializing cgroup subsys net_prio<br>[ 0.004000] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'<br>ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)<br>[ 0.004000] CPU: Physical Processor ID: 0<br>[ 0.004000] CPU: Processor Core ID: 1<br>[ 0.004000] Last level iTLB entries: 4KB 512, 2MB 8, 4MB 8<br>Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0<br>tlb_flushall_shift: 2<br>[ 0.025772] Freeing SMP alternatives memory: 20K (ffffffff81a1b000 - ffffffff81a20000)<br>[ 0.028385] ftrace: allocating 21679 entries in 85 pages<br>[ 0.036140] Performance Events: unsupported p6 CPU model 58 no PMU driver, software events only.<br>[ 0.037591] x86: Booted up 1 node, 1 CPUs<br>[ 0.037647] NMI watchdog: disabled (cpu0): hardware events not enabled<br>[ 0.037724] devtmpfs: initialized<br>[ 0.041230] pinctrl core: initialized pinctrl subsystem<br>[ 0.041317] NET: Registered protocol family 16<br>[ 0.041334] xen:grant_table: Grant tables using version 1 layout<br>[ 0.041353] Grant table initialized<br>[ 0.041615] PCI: setting up Xen PCI frontend stub<br>[ 0.041615] PCI: pci_cache_line_size set to 64 bytes<br>[ 0.041615] ACPI: Interpreter disabled.<br>[ 0.041615] xen:balloon: Initialising balloon driver<br>[ 0.044041] xen_balloon: Initialising balloon driver<br>[ 0.044067] vgaarb: loaded<br>[ 0.044134] PCI: System does not support PCI<br>[ 0.044138] PCI: System does not support PCI<br>[ 0.044274] Switched to clocksource xen<br>[ 0.049387] pnp: PnP ACPI: disabled<br>[ 0.053177] NET: Registered protocol family 2<br>[ 0.053357] TCP established hash table entries: 1024 (order: 1, 8192 bytes)<br>[ 0.053368] TCP bind hash table entries: 1024 (order: 2, 16384 bytes)<br>[ 0.053373] TCP: Hash tables configured (established 1024 bind 1024)<br>[ 0.053389] TCP: reno registered<br>[ 0.053394] UDP hash table entries: 256 (order: 1, 8192 bytes)<br>[ 0.053402] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)<br>[ 0.053439] NET: Registered protocol family 1<br>[ 0.053454] PCI: CLS 0 bytes, default 64<br>[ 0.053501] Unpacking initramfs...<br>[ 0.091622] Freeing initrd memory: 43680K (ffff880001f18000 - ffff8800049c0000)<br>[ 0.093485] platform rtc_cmos: registered platform RTC device (no PNP device found)<br>[ 0.093788] RAPL PMU detected, hw unit 2^-16 Joules, API unit is 2^-32 Joules, 3 fixed counters 163840 ms ovfl timer<br>[ 0.093845] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x19<br>[ 0.093925] microcode: Microcode Update Driver: v2.00 <<a href="mailto:tigran@aivazian.fsnet.co.uk">tigran@aivazian.fsnet.co.uk</a>>, Peter Oruba<br>[ 0.094168] futex hash table entries: 256 (order: 2, 16384 bytes)<br>[ 0.094197] audit: initializing netlink subsys (disabled)<br>[ 0.094217] audit: type=2000 audit(1454158364.375:1): initialized<br>[ 0.094532] HugeTLB registered 2 MB page size, pre-allocated 0 pages<br>[ 0.094557] zbud: loaded<br>[ 0.094700] VFS: Disk quotas dquot_6.5.2<br>[ 0.094714] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)<br>[ 0.094771] msgmni has been set to 227<br>[ 0.095221] alg: No test for stdrng (krng)<br>[ 0.095249] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)<br>[ 0.095292] io scheduler noop registered (default)<br>[ 0.095297] io scheduler deadline registered<br>[ 0.095324] io scheduler cfq registered<br>[ 0.095389] pci_hotplug: PCI Hot Plug PCI Core version: 0.5<br>[ 0.095407] pciehp: PCI Express Hot Plug Controller Driver version: 0.4<br>[ 0.095437] intel_idle: does not run on family 6 model 58<br>[ 0.095735] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled<br>[ 0.095989] Linux agpgart interface v0.103<br>[ 0.096104] i8042: PNP: No PS/2 controller found. Probing ports directly.<br>[ 1.270619] i8042: No controller found<br>[ 1.270793] mousedev: PS/2 mouse device common for all mice<br>[ 1.348656] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0<br>[ 1.348710] rtc_cmos: probe of rtc_cmos failed with error -38<br>[ 1.348741] ledtrig-cpu: registered to indicate activity on CPUs<br>[ 1.348760] AMD IOMMUv2 driver by Joerg Roedel <<a href="mailto:joerg.roedel@amd.com">joerg.roedel@amd.com</a>><br>[ 1.348763] AMD IOMMUv2 functionality not available on this system<br>[ 1.348859] TCP: cubic registered<br>[ 1.348876] NET: Registered protocol family 10<br>[ 1.349056] mip6: Mobile IPv6<br>[ 1.349064] NET: Registered protocol family 17<br>[ 1.349073] mpls_gso: MPLS GSO support<br>[ 1.349080] mce: Unable to init device /dev/mcelog (rc: -5)<br>[ 1.349225] registered taskstats version 1<br>[ 1.349283] xenbus_probe_frontend: Device with no driver: device/vbd/51714<br>[ 1.349288] xenbus_probe_frontend: Device with no driver: device/vbd/51713<br>[ 1.349291] xenbus_probe_frontend: Device with no driver: device/vif/0<br>[ 1.349294] xenbus_probe_frontend: Device with no driver: device/pci/0<br>[ 1.349331] rtc_hctosys: unable to open rtc device (rtc0)<br>[ 1.349380] PM: Hibernation image not present or could not be loaded.<br>[ 1.349896] Freeing unused kernel memory: 1204K (ffffffff818ee000 - ffffffff81a1b000)<br>[ 1.349907] Write protecting the kernel read-only data: 8192k<br>[ 1.352430] Freeing unused kernel memory: 912K (ffff88000151c000 - ffff880001600000)<br>[ 1.352541] Freeing unused kernel memory: 212K (ffff8800017cb000 - ffff880001800000)<br>[ 1.372341] systemd-udevd[58]: starting version 215<br>[ 1.372888] random: systemd-udevd urandom read with 3 bits of entropy available<br>[ 1.395569] xen_netfront: Initialising Xen virtual ethernet driver<br>[ 1.443740] blkfront: xvda2: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;<br>[ 1.445979] blkfront: xvda1: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;<br>[ 8.862010] EXT4-fs (xvda2): mounted filesystem with ordered data mode. Opts: (null)<br>[ 9.351477] random: nonblocking pool is initialized<br>[ 10.447043] systemd[1]: systemd 215 running in system mode. (+PAM +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ -SECCOMP -APPARMOR)<br>[ 10.447128] systemd[1]: Detected virtualization 'xen'.<br>[ 10.447139] systemd[1]: Detected architecture 'x86-64'.<br>[ 10.912912] systemd[1]: Inserted module 'autofs4'<br>[ 10.971682] systemd[1]: Set hostname to <bug>.<br>[ 15.114090] systemd[1]: Cannot add dependency job for unit dbus.socket, ignoring: Unit dbus.socket failed to load: No such file or directory.<br>[ 15.114125] systemd[1]: Cannot add dependency job for unit display-manager.service, ignoring: Unit display-manager.service failed to load: No such file or directory.<br>[ 15.114652] systemd[1]: Starting Forward Password Requests to Wall Directory Watch.<br>[ 15.114803] systemd[1]: Started Forward Password Requests to Wall Directory Watch.<br>[ 15.114834] systemd[1]: Expecting device dev-hvc0.device...<br>[ 15.115023] systemd[1]: Starting Remote File Systems (Pre).<br>[ 15.115131] systemd[1]: Reached target Remote File Systems (Pre).<br>[ 15.115185] systemd[1]: Starting Arbitrary Executable File Formats File System Automount Point.<br>[ 15.115422] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.<br>[ 15.115463] systemd[1]: Starting Dispatch Password Requests to Console Directory Watch.<br>[ 15.115547] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.<br>[ 15.115571] systemd[1]: Starting Paths.<br>[ 15.115669] systemd[1]: Reached target Paths.<br>[ 15.115692] systemd[1]: Starting Encrypted Volumes.<br>[ 15.115785] systemd[1]: Reached target Encrypted Volumes.<br>[ 15.115808] systemd[1]: Expecting device dev-xvda1.device...<br>[ 15.115899] systemd[1]: Starting Root Slice.<br>[ 15.116040] systemd[1]: Created slice Root Slice.<br>[ 15.116064] systemd[1]: Starting User and Session Slice.<br>[ 15.116207] systemd[1]: Created slice User and Session Slice.<br>[ 15.116231] systemd[1]: Starting Delayed Shutdown Socket.<br>[ 15.116365] systemd[1]: Listening on Delayed Shutdown Socket.<br>[ 15.116389] systemd[1]: Starting /dev/initctl Compatibility Named Pipe.<br>[ 15.116524] systemd[1]: Listening on /dev/initctl Compatibility Named Pipe.<br>[ 15.116548] systemd[1]: Starting Journal Socket (/dev/log).<br>[ 15.116688] systemd[1]: Listening on Journal Socket (/dev/log).<br>[ 15.116721] systemd[1]: Starting udev Kernel Socket.<br>[ 15.116829] systemd[1]: Listening on udev Kernel Socket.<br>[ 15.116859] systemd[1]: Starting udev Control Socket.<br>[ 15.116970] systemd[1]: Listening on udev Control Socket.<br>[ 15.116997] systemd[1]: Starting Journal Socket.<br>[ 15.117135] systemd[1]: Listening on Journal Socket.<br>[ 15.117170] systemd[1]: Starting System Slice.<br>[ 15.117300] systemd[1]: Created slice System Slice.<br>[ 15.117341] systemd[1]: Starting File System Check on Root Device...<br>[ 15.118784] systemd[1]: Starting system-getty.slice.<br>[ 15.118980] systemd[1]: Created slice system-getty.slice.<br>[ 15.119005] systemd[1]: Starting system-serial\x2dgetty.slice.<br>[ 15.119146] systemd[1]: Created slice system-serial\x2dgetty.slice.<br>[ 15.119224] systemd[1]: Starting Increase datagram queue length...<br>[ 15.120664] systemd[1]: Mounting POSIX Message Queue File System...<br>[ 15.315723] systemd[1]: Mounting Debug File System...<br>[ 15.376731] systemd[1]: Started Set Up Additional Binary Formats.<br>[ 15.376795] systemd[1]: Mounting Huge Pages File System...<br>[ 15.378081] systemd[1]: Starting Create list of required static device nodes for the current kernel...<br>[ 15.422179] systemd[1]: Starting Load Kernel Modules...<br>[ 15.423348] systemd[1]: Starting udev Coldplug all Devices...<br>[ 15.424454] systemd[1]: Starting Slices.<br>[ 15.424548] systemd[1]: Reached target Slices.<br>[ 15.425330] systemd[1]: Started Increase datagram queue length.<br>[ 15.426545] systemd[1]: Starting Syslog Socket.<br>[ 15.426671] systemd[1]: Listening on Syslog Socket.<br>[ 15.426694] systemd[1]: Starting Sockets.<br>[ 15.426763] systemd[1]: Reached target Sockets.<br>[ 15.426783] systemd[1]: Starting Journal Service...<br>[ 15.429296] systemd[1]: Started Journal Service.<br>[ 16.279394] systemd-udevd[137]: starting version 215<br>[ 16.420090] EXT4-fs (xvda2): re-mounted. Opts: errors=remount-ro<br>[ 16.640700] systemd-journald[123]: Received request to flush runtime journal from PID 1<br>[ 17.378097] input: PC Speaker as /devices/platform/pcspkr/input/input0<br>[ 17.384721] pcifront pci-0: Installing PCI frontend<br>[ 17.384741] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB<br>[ 17.387512] software IO TLB [mem 0x02400000-0x02800000] (4MB) mapped at [ffff880002400000-ffff8800027fffff]<br>[ 17.388459] pcifront pci-0: Creating PCI Frontend Bus 0000:00<br>[ 17.388562] pcifront pci-0: PCI host bridge to bus 0000:00<br>[ 17.388569] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]<br>[ 17.388573] pci_bus 0000:00: root bus resource [mem 0x00000000-0xfffffffff]<br>[ 17.388578] pci_bus 0000:00: root bus resource [bus 00-ff]<br>[ 17.401449] pci 0000:00:01.0: [1912:0014] type 00 class 0x0c0330<br>[ 17.401911] pci 0000:00:01.0: reg 0x10: [mem 0xfbff0000-0xfbff1fff 64bit]<br>[ 17.431844] alg: No test for crc32 (crc32-pclmul)<br>[ 17.435526] pci 0000:00:00.1: [14e4:165f] type 00 class 0x020000<br>[ 17.435907] pci 0000:00:00.1: reg 0x10: [mem 0xfabc0000-0xfabcffff 64bit pref]<br>[ 17.436227] pci 0000:00:00.1: reg 0x18: [mem 0xfabb0000-0xfabbffff 64bit pref]<br>[ 17.436481] pci 0000:00:00.1: reg 0x20: [mem 0xfaba0000-0xfabaffff 64bit pref]<br>[ 17.437994] pcifront pci-0: New device on 0000:00:00.1 found.<br>[ 17.439770] pcifront pci-0: claiming resource 0000:00:01.0/0<br>[ 17.439778] pcifront pci-0: claiming resource 0000:00:00.1/0<br>[ 17.439782] pcifront pci-0: claiming resource 0000:00:00.1/2<br>[ 17.439785] pcifront pci-0: claiming resource 0000:00:00.1/4<br>[ 17.439875] pci 0000:00:01.0: Xen PCI mapped GSI18 to IRQ29<br>[ 17.742850] intel_rapl: Found RAPL domain package<br>[ 17.742864] intel_rapl: Found RAPL domain core<br>[ 18.330425] pps_core: LinuxPPS API ver. 1 registered<br>[ 18.330438] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <<a href="mailto:giometti@linux.it">giometti@linux.it</a>><br>[ 18.335293] PTP clock support registered<br>[ 18.382328] tg3.c:v3.137 (May 11, 2014)<br>[ 18.383508] tg3 0000:00:00.1: Xen PCI mapped GSI17 to IRQ30<br>[ 18.394807] usbcore: registered new interface driver usbfs<br>[ 18.395074] usbcore: registered new interface driver hub<br>[ 18.395796] usbcore: registered new device driver usb<br>[ 18.413270] tg3 0000:00:00.1 eth1: Tigon3 [partno(N/A) rev 5720000] (PCI Express) MAC address 34:64:a9:9a:a8:b1<br>[ 18.413288] tg3 0000:00:00.1 eth1: attached PHY is 5720C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])<br>[ 18.413297] tg3 0000:00:00.1 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1]<br>[ 18.413304] tg3 0000:00:00.1 eth1: dma_rwctrl[00000001] dma_mask[64-bit]<br>[ 18.430176] xen:events: xen_bind_pirq_gsi_to_irq: returning irq 29 for gsi 18<br>[ 18.430192] xhci_hcd 0000:00:01.0: Xen PCI mapped GSI18 to IRQ29<br>[ 18.434698] xhci_hcd 0000:00:01.0: xHCI Host Controller<br>[ 18.434723] xhci_hcd 0000:00:01.0: new USB bus registered, assigned bus number 1<br>[ 18.445842] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002<br>[ 18.445856] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1<br>[ 18.445862] usb usb1: Product: xHCI Host Controller<br>[ 18.445867] usb usb1: Manufacturer: Linux 3.16.0-4-amd64 xhci_hcd<br>[ 18.445872] usb usb1: SerialNumber: 0000:00:01.0<br>[ 18.449363] hub 1-0:1.0: USB hub found<br>[ 18.449392] hub 1-0:1.0: 2 ports detected<br>[ 18.449530] xhci_hcd 0000:00:01.0: xHCI Host Controller<br>[ 18.449540] xhci_hcd 0000:00:01.0: new USB bus registered, assigned bus number 2<br>[ 18.450543] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.<br>[ 18.450584] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003<br>[ 18.450591] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1<br>[ 18.450596] usb usb2: Product: xHCI Host Controller<br>[ 18.450602] usb usb2: Manufacturer: Linux 3.16.0-4-amd64 xhci_hcd<br>[ 18.450606] usb usb2: SerialNumber: 0000:00:01.0<br>[ 18.450904] hub 2-0:1.0: USB hub found<br>[ 18.450925] hub 2-0:1.0: 2 ports detected<br>[ 18.532710] Adding 262140k swap on /dev/xvda1. Priority:-1 extents:1 across:262140k SSFS<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Looking at the code (in Linux) I see:<br>
516 /* NB. We are happy to share unless we are probing. */<br>
517 bind_pirq.flags = info->u.pirq.flags & PIRQ_SHAREABLE ?<br>
518 BIND_PIRQ__WILL_SHARE : 0;<br>
519 rc = HYPERVISOR_event_channel_op(EVTCHNOP_bind_pirq, &bind_pirq);<br>
520 if (rc != 0) {<br>
521 pr_warn("Failed to obtain physical IRQ %d\n", irq);<br>
522 return 0;<br>
523 }<br>
<br>
It would be nice if I we had included the return code :-(<br>
<br>
Anyhow for PV guests in the hypervisor there is a check:<br>
414 if ( !is_hvm_domain(d) && !pirq_access_permitted(d, pirq) )<br>
415 return -EPERM;<br>
416<br>
<br>
And I wonder if the XEN_DOMCTL_irq_permission aka, xc_domain_irq_permission<br>
had been called. I remember that at some point we missed it for Xend..<br>
<br>
Could you enable Xend debugging (if you are using that):<br></blockquote><div><br></div><div>I'm using xl. Is there a similar setting for that?<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
export XEND_DEBUG=1<br>
/etc/init.d/xend start<br>
<br>
And try starting the guest. Then grep in /var/log/xen/xend.log for<br>
"pci: enabling irq "<br>
<br>
You should see 18, 17 and 21.<br>
<br>
Also if you do 'xl debug-keys i && xl dmesg'<br>
<br>
You should see that the IRQ 17, 18, 21 are assigned to the domain.<br>
?<br>
</blockquote></div><br></div><div class="gmail_extra">Br,<br></div><div class="gmail_extra">Tommi Airikka<br></div><div class="gmail_extra"><br></div></div>