XPS 13 9360 trackpad locks into scroll mode

Thanks for raising this issue. I currently have a Dell XPS 9360 as a loaner so the timing is perfect. I haven't used it enough to trigger the issue on a vanilla kernel however.

Given that the scroll lock happens even if the hid-multitouch and i2c-hid modules are removed and re-added, I would *think* the issue is in user space, not in kernel space. The reason is that the last data in comment #110 show that the kernel is behaving properly by rejecting the ghost touches on my system if I replay the data.

Once the bug is triggered, could you ask the user who triggered it to rmmod i2c-hid and modprobe it back with debug=1, and attach the output of dmesg here? This should show the exact starting sequence of the bug, and we will be able to rule out if it is a kernel or a user space issue.

Created attachment 260527
Scroll lock that vanishes very fast

My touchpad is in the state again that I can trigger the bug deliberately, but not a really nasty lock, only for a few seconds.

I started to load the module with debug mode, started scrolling with two fingers, released one finger and it kept scrolling. I even removed the finger touched again to move the cursor but it still was in scroll mode. The issue resolved itself again without any click.

Hope you can find something in there, at the moment the locks only last very shortly for me.

Created attachment 260545
kern.log from events in the video

The touchpad started misbehaving again, but not as badly as it could be.
I started to film the problem, see

https://www.youtube.com/watch?v=HpAWC8imIcc

In the video I first show off the scroll lock happening, then I run a script that reloads the i2c-hid module with debug info and shows a timestamp every second, for you to get a better understanding of what happened at which point in time in the attached kern.log file.

You can also see the mouse cursor jumping at 0:50.

Thanks for both logs and the video. This makes things easier to see and understand.

The first log did show a ghost touch on the upper right corner of the touchpad, but I wasn't sure if you accidentally touched this part of the touchpad or not. With the video, I can see that you are only using one finger on the touchpad, and that the touch that regularly appears on the top right corner is a ghost one.

As far as I can tell, there is no (simple) way from the drier to ignore those. The firmware marks those touches are valid, and it would require far too much processing to eliminate them directly from the driver.

On the 9360 I have been loaned, I do not see such ghosts. Gnome software says that my bios is 0.1.3.5 and that there is an update to 0.2.3.1. Which Bios are you? There might have been an upgrade of the touchpad that broke it. If you do not have gnome-software with fwupd, you can just run "sudo dmidecode -s bios-version" to get the version.

Created attachment 260547
recording from the video, in the hid-replay format

Just so I do not lose it later, here is the converted data I extracted from the second dmesg. You can replay the data and parse it into a human readable format with https://github.com/bentiss/hid-replay

I am using bios 2.2.1. I have seen this issue in different bios versions.

Comment #131 in launchpad says:

"For this particular issue, this information is not useful however because the firmware for the XPS 9360 touchpad is unchanged since it's been on the market."

So I guess the bios version is not relevant!?

I am sure you are having the issue too, it just takes time to see it. When you use the 9360 as your daily driver you will see the issue for sure. It could take a few weeks to first happen, but it does happen unfortunately :-(.

I am speaking out of experience. There can be a long time where the touchpad behaves perfectly and suddenly it goes crazy.

Thanks very much for looking into this. Anything more we could do!?

@Benjamin thank you for your help on this issue! Your patches to hid have made you a bit of a hero to people with this issue, so I'm just excited to post on a thread with you. :)

I am using bios version 1.3.5 with the same problem (I'm the one who provided the event dumps in the launchpad thread).

I'm currently of the mindset that this is a hardware issue. The last week or so, I've been "curing" my attacks with firm pressure on the space between the trackpad and the space bar. Firm enough to risk a physical click on the touchpad. This seems to solve the episodes for a longer time than my previous method (hard physical click on the right side of the touchpad).

That said, it's only been a week. I've thought I've had a solution before.

Looking at a disassembly of the laptop ( https://www.parts-people.com/blog/wp-content/uploads/2016/12/xps9350touchpadconnector.jpg ) though, I don't see what I could be pressing on. The touchpad connetor is in the middle of the touchpad. Maybe one of the pads that holds it in place is a little too far over and generating a click? I always thought touchpads worked with capacitance though, and it would be a pretty dumb material choice for padding if it changes apparent capacitance where it touches the trackpad. Also, that wouldn't explain the "flickering" effect.

I second Esokrarkose's comment - try using the laptop as a daily driver. I'm sure you'll run into the issue eventually. It very rarely occurs when I start using the machine, usually it pops up over time.

Right. So if Windows is immune to this problem, there must be some sort of palm rejection mechanism in user space that prevents it to see the issue. I'll check with Peter Hutterer if this is something doable.

Answer from Peter: https://bugs.freedesktop.org/show_bug.cgi?id=103208
Once this bug is fixed in libinput, you should be fine. But note that this is a firmware issue still :)

Some users reported that Windows is also affected [1] in comment #3, #21.

[1] https://bugs.launchpad.net/bugs/1651635

Regarding Windows being affected: I did some research, but nowhere outside of the launchpad page I could find someone confirming it is an issue there too and I guess there are much more XPS 13 Windows users out there than linux users, but since it is an firmware issue I guess the two comments may be right, still I am a bit surprised it was not mentioned somewhere outside the linux userers world, maybe it's less likely to appear there?

Mario Limonciello from Dell is subscribed here, maybe he could comment whether it is likely that a firmware update is issued?

Benjamin Tissoires: Shall I keep recording the issue to find out if it's always the upper right corner where the ghost touches are coming from?

can you get me an evemu-record output for the ghost touch please? Is it always in the same spot, or roughly the same spot?

When it happens again and I have the time I will upload the evemu-record along with the other debug info.

Created attachment 260565
kern log of the stuck pointer in the video

Touchpad got stuck, the pointer locked completely, I remember dragging one finger from outside the upper right corner to the touchpad when the pointer got stuck. Then I started recording the video:

https://youtu.be/GXw0MVjbn2w

As last time, the script I run reloaded the kernel module with debug info, as can be seen, the pointer was still stuck. The lock was released in 0:51, when I pressed the right button, as can be heard in the video.

The evemu-record will be attached in the next comment.

Created attachment 260567
evemu-record output of video, as requested

evemu-record output of https://youtu.be/GXw0MVjbn2w

Created attachment 260569
kern log of right corner triggering

As you told me the upper right corner might trigger this I played around and found I could trigger misbehavior by playing around a bit (but this only works sometimes, not always). I recorded that experience once again:

https://youtu.be/gEYBBP_6Yr0

Right now as I am typing the touchpad is also misbehaving, but I do not have unlimited time to record all the time. I can't tell if it's always related to the upper right corner.

Created attachment 260571
evemu-record of upper right corner triggering

evemu-record of https://www.youtube.com/watch?v=gEYBBP_6Yr0

If you need snapshots of the evemu-record.log my script took every second to diff what happened at every second in the video let me know. Also available for the prior evemu-record.

Created attachment 260573
lower corner kern.log

One last time, triggering scroll lock with right lower corner and then with the right upper corner.

Watch closely: In the beginning I trigger a scroll lock with the lower corner.

https://youtu.be/_O6BJQYYY7c

Sometimes though the scroll locking happens although I do not remember getting near a corner, at least I think so, I am not sure.

Created attachment 260575
lower corner evemu recording

Corresponding evemu recording for https://youtu.be/_O6BJQYYY7c.

A few clarification points and thoughts:

1) On the Windows side there is no extra Synaptics driver doing any palm rejection. Windows uses the inbox HID driver. Anything handled in this area would be entirely with the built-in Windows software stack.

2) There haven't been any touchpad firmware updates for the XPS 9360 since launch.

3) Upgrading system firmware (BIOS) I don't believe should have any bearing on this particular issue. I know BIOS update flashes a lot of components, but touchpad is not one of them.

>Mario Limonciello from Dell is subscribed here, maybe he could comment
>>whether it is likely that a firmware update is issued?

As of right now there aren't any touchpad firmware updates in the works.

Although there could be some potential improvements to palm rejection handling in libinput as mentioned above, I'm highly suspicious of a few pieces of defective hardware for those afflicted by this annoying issue. And yes I know that one person on Launchpad mentioned they had TP replaced but issue came back, but it could still be a defective replacement too.

@esokrarkose, since you seem to be able to readily reproduce on your piece of hardware I think it would be ideal if you could set up a dual boot or a Windows to go USB stick (If you have access) that you can experiment with Windows 10's inbox driver. If you can still reproduce problems there I'm going to go on a limb and say it really is defective hardware and not something that should be bandaged by libinput to look for the ghost touches.

Evidence of it happening in both places should make a pretty strong case for repair.

"a few pieces of defective hardware for those afflicted by this annoying issue"

That comment is making me quite angry. I am not wasting a lot of time so that it gets rejected by saying: this only affects a few pieces of hardware implying it is not relevant. I spent money for two pieces of this hardware manufactured at different dates (more than a month inbetween) and both have the issue.

Paul Menzel also subscribed to this issue has colleagues with the same problem. One person got a replacement and the issue was still there. A very very unlikely coincidence, isn't it? It's likely that MOST of the touchpads are broken then, if you do not believe that all are affected.

When I have more time I will have a look at Windows, but I have already wasted too much time with that, I do not expect such problems for that price tag.

I'm sorry, but I don't have statistics to show number of calls related to touchpad relative to units sold. I don't work in the Dell support organization. I'm trying to add to this issue context that will help get down to a root cause.

My suspicion in it being defective hardware is colored by the fact that I've used both an XPS 9350 and XPS 9360 daily (Which each contain Synaptics I2C TP) for a while now and never experienced this issue myself and know dozens of others who have as well.

As you mentioned: it's also suspicious that a handful of people have the problem, they seem to all be on Linux, and they have congregated on a Launchpad bug. This is the internet, people with similar views, similar problems flock to the same place.

Even if it's defective hardware i'm not saying it can't be helped by libinput adjustments. I'm saying the surface area of afflicted people (appears) small so a hardware problem is a potential root cause and repair may be the quickest solution.

looking at the logs in attachment 260547:
* the ghost touch starts at 1216/0 (xmax/ymin) and is constant
* at 875.329542 it is at 1192/14
* It later *moves* to 1185/18 over several events and stays there. This pattern repeats multiple times

The highest distance from the edge is 1185 and 20, so roughly 3 and 2 mm, respectively. Note: this is from a manual analysis, not a scripted one so I may have missed a value. I'm assuming the others are the same, because in the end they can only be worse, if they're better than the above that's great ;)

For userspace this means:
* there isn't a single coordinate that we can blacklist
* we need some heuristics beyond 'maximum edge of the touchpad' because 3/2mm is a significant distance in. that heuristic runs the danger of interfering with real touches
* we can't just look at a touch and ignore it when it doesn't move - some of them move. Although it looks like they move late enough that we can pick them out before that happens.

Either way, the above would definitely require a model-specific quirk because we don't want to affect other devices. I wouldn't be surprised if windows papers over this accidentally, in the same way as the bug linked in comment 10 would probably paper over this. The windows bit is a guess though.

I'm going with Mario here and suspect this is defective hardware. If every XPS user would have this issue it'd be a different matter but fixing this has a nonzero cost for userspace.

Voicing my experience here - as someone who thinks it's a hardware issue... I dual boot this machine and play games on the Windows side. I've never had it appear in Windows 10. Also, the couple of comments in Launchpad are the only reports I can find _anywhere on the Internet_ of it happening in Win.

There are many reports from users who have had the touchpad replaced, or reseated. They think it's OK for a week or two, then it returns. I've even read threads from people who have gone through 3 or more replacements this way. :(

I'm inclined to think it's something that the windows driver accidentally papers over. Is the touchpad size detection in Win the same? Is there potentially a small dead zone around the outside, or are we potentially detecting a larger surface than exists?

> Is there potentially a small dead zone around the outside, or are we
> potentially detecting a larger surface than exists?

Thats quite easy to verify, just run the touchpad-edge-detector tool

I used the measurement from Dell's specs on their website for my service tag number.

```
sudo touchpad-edge-detector 105x60 /dev/input/event16

Touchpad DLL075B:01 06CB:76AF Touchpad on /dev/input/event16
Move one finger around the touchpad to detect the actual edges
Kernel says: x [0..1216], y [0..680]
Touchpad sends: x [3..1216], y [0..680] -|-| \^C

Touchpad size as listed by the kernel: 101x56mm
User-specified touchpad size: 105x60mm
Calculated ranges: 1213/680

Suggested udev rule:
# <Laptop model description goes here>
evdev:name:DLL075B:01 06CB:76AF Touchpad:dmi:bvnDellInc.:bvr1.3.5:bd05/08/2017:svnDellInc.:pnXPS139360:pvr:rvnDellInc.:rn06CC14:rvrA00:cvnDellInc.:ct9:cvr:*
 EVDEV_ABS_00=3:1216:12
 EVDEV_ABS_01=0:680:11
 EVDEV_ABS_35=3:1216:12
 EVDEV_ABS_36=0:680:11
```

Looks like the kernel expected a smaller device. Not sure what to make of that, but I'll give it a try and report back! This week has been pretty calm for me as far as this problem goes, so I'd be interested to see the output/effect of touchpad-edge-detector settings from someone else...

It takes a while until it matches what Kernel and Touchpad say, but here it is:

sudo touchpad-edge-detector 105x60 /dev/input/event10
Touchpad DLL075B:01 06CB:76AF Touchpad on /dev/input/event10
Move one finger around the touchpad to detect the actual edges
Kernel says: x [0..1216], y [0..680]
Touchpad sends: x [0..1216], y [0..680] \^C\\|

Touchpad size as listed by the kernel: 101x56mm
User-specified touchpad size: 105x60mm
Calculated ranges: 1216/680

Suggested udev rule:
# <Laptop model description goes here>
evdev:name:DLL075B:01 06CB:76AF Touchpad:dmi:bvnDellInc.:bvr2.2.1:bd08/18/2017:svnDellInc.:pnXPS139360:pvr:rvnDellInc.:rn05JK94:rvrA00:cvnDellInc.:ct9:cvr:*
 EVDEV_ABS_00=0:1216:12
 EVDEV_ABS_01=0:680:11
 EVDEV_ABS_35=0:1216:12
 EVDEV_ABS_36=0:680:11

(In reply to Esokrarkose from comment #22)
> "a few pieces of defective hardware for those afflicted by this annoying
> issue"
>
> That comment is making me quite angry. I am not wasting a lot of time so
> that it gets rejected by saying: this only affects a few pieces of hardware
> implying it is not relevant. I spent money for two pieces of this hardware
> manufactured at different dates (more than a month inbetween) and both have
> the issue.
>
> Paul Menzel also subscribed to this issue has colleagues with the same
> problem.

Just a clarification, that we couldn’t reproduce the issue in Microsoft Windows 10 (we reluctantly installed and took some time). After that, we were not able to reproduce it with an Ubuntu installation anymore.

> When I have more time I will have a look at Windows, but I have already
> wasted too much time with that, I do not expect such problems for that price
> tag.

I totally agree, that Dell’s QA failed big time. But please try to reproduce the issue with Microsoft Windows, and to try to get a replacement for both devices.

FWIW, touchpad-edge-detector recommends the exact same values on the XPS 9350 as the values recommended by comment #28 and comment #27.
evdev:name:DLL0704:01 06CB:76AE

"Just a clarification, that we couldn’t reproduce the issue in Microsoft Windows 10 (we reluctantly installed and took some time). After that, we were not able to reproduce it with an Ubuntu installation anymore."

Right, ok then you still have the issue, it will return eventually, because it wasn't fixed magically.

Yesterday and by the start of today the issue was bugging me extremly, now its all of the sudden gone, I can't trigger it, no matter how hard I try, even the right upper and lower edges behave correctly. There was no reboot inbetween just a couple of suspends.

It's extremely frustrating to convince you guys, but at the end of the day that touchpad is simply unreliable, there is definitely an issue and it will return at a random point.

I mean I'm okay with changing the touchpad, I would even request the guys from Dell to do so, but I am 100% certain, that the problem will return.

I am frustrated, because if I would carry my computer to you guys and try to show the problem it wouldn't work instantly almost certainly. If you would spend a day with me, I could show you when it happens just like in the video. I am sure the same would happen to me when Mario would give me his XPS 13 model. Maybe it's related to usage pattern!?

(In reply to Campbell Vertesi from comment #27)
> Looks like the kernel expected a smaller device.

Rounding error, not to worry about. Kernel provides the resolution in
units/mm as integer, but your touchpad has 11.58 u/mm. Rounding up the
resolution to 12 results in a perceived smaller size (101 vs 105mm). This
doesn't usually matter unless it's out by some significant amount.

??Hello,
I will be out of office 11/10 returning 11/13. Expect delayed email responses.

Thanks @Peter for the explanation.

@Esokrarkose the next time this happens to you.. could you please try putting pressure on some other parts of the case, and see if that resolves it for you? I've found that firm pressure in the space between the trackpad and the space bar resolves it for me. Can you replicate?

Damn, pressing on the case between the right corner and "Alt Gr" triggered the issue. When moving the finger on the case with pressure (not touching the touchpad) the mouse pointer sometimes moves very slightly :-(.

Windows 10 also affected. It behaves a little bit better, but I could reproduce the jumping and the lock.

Can others who have the same issue verify that their models have a weak spot on the case between upper right corner and "Alt Gr" key and pressing there makes the trackpad go mad?

Campbell, when was your system manufactured, can you also trigger the issue pressing on the spot I suggest?

Mario, could you somehow help me convince support that it's been verified that a swap is justified in my case? All the stuff they want me to try is obviously just a waste of time.

(In reply to Esokrarkose from comment #37)

Hi I have an XPS 13 9360 with the touchpad issue but do not have such a weak spot. Pressing the case between upper right corner and the right Alt key doesn't do a thing.

Well, it sucks to discover it's a hardware issue for sure. :(

I bought my system in June 2017. Service tag # C3L06H2 so you can look up the specs :)

I don't have a particular weak spot between the trackpad and the right alt key. I can press anywhere above the trackpad and it ends an episode, but I can't seem to trigger an episode that way. I'm continuing to experiment with different spots on the below-keyboard part of the case to see if I can figure out a pattern.

Given that it seems very forgiving about precisely where I press, right now my theory is a charge buildup from improper grounding somewhere.

Further discussion of this really doesn't belong on the kernel bugzilla, but I don't know where else to do it. Is it OK if we continue debugging the issue in this thread?

I'm curious to hear, anyone affected who has used the output from touchpad-edge-detector, did you have any positive results?

I believe that's the only potential software change on the table right now as an outcome of this bug. We should probably close it after deciding that.

@esokrakose, Campbell (and any others that this is looking like a HW issue):
Regarding where to take this discussion of a potential HW issue:
You can bring it to the Dell project sputnik forums to openly discuss as a group: http://en.community.dell.com/techcenter/os-applications/f/4613

If you're having a hard time with the phone support group, I'm sorry about that. I talked to one of our web support guys and pre-wired them on the situation going on. You can join the Dell forums and send him a friend request with your service tag through his profile:
http://en.community.dell.com/members/dell_2d00_justin-c

This can be closed, it is indeed a hardware issue. Got my touchpad replaced in November and I have never experienced this again ever since. Also the Dell support guys on twitter are amazing!