Month in Review: November 2017

As you will notice, I came within a hair's (well, a day's!) breadth of missing my two post target! Posting two days in a row isn't brilliant spacing, but never mind! Such is the nature of November: this November in particular.

This is one of those periods of "peak teach" I've discussed before. Projects are in full swing; early assignments are in for marking, later assignments are being finalised and set; exams are due (I wrote mine over the summer, but a late change in regs meant writing an extra question at short notice!); lectures and tutorials must be delivered; applicant days have begun. On top of that, we had an accreditation meeting this week. And no less than three seminars at the University have cropped up.

All good, but it all means a fine balancing act. A new grant due to start in January means finalising budgets, advertising jobs, and booking a trip to Sweden. More on that in due course.

Amongst all this, there have been three significant research events. The grand finale of the Tracking People AHRC network ran at the start of November - I wrote about my thoughts on this in the previous post, so I won't go further here.

On the Apex grant that Stuart Murray and I are working on, we've just had a copy of the Ada hand printed, which is really exciting. And my PhD student, Latif Azyze, has just some exciting results on handwriting forces.

Busy, busy, busy - but all good stuff. Roll on December, eh?

Tracking People: The Grand Finale

As you’ll know if you’ve been following me on Twitter, this month saw the culmination of the AHRC Network on Tracking People: a day-long workshop down in London, bringing together a range of industrialists, civil servants, academics and representatives of a range of third sector organisations. There were more than eighty attendees: familiar faces from the three workshops at Leeds, and a range of new faces (the rationale for holding it in London was that it made attendance for policymakers far easier – a move which I think paid off). Policy is a fascinating issue to me: it’s so far outside my usual experiences, so it was interesting to speak with representatives of the Centre for Applied Science and Technology, who advise the government on matters technical.
    The day was generally given over to discussion: opening with summaries from Anthea Hucklesby, Kevin Macnish and myself reflecting on insights from the previous three workshops and what these might mean for the future of “tracking”. You can read summaries from these workshops on the tracking website, and I’ll give a “big picture” overview of my thoughts from the network as a whole in a later post. For now, let me focus on the content of the presentations.
     The first session saw Jeff Hodgkinson (lately of South Wales Police) and Sara Murray of Buddi give practitioners’ perspectives. Jeff spoke of his experiences of electronic monitoring in criminal justice, and the need for more joined up and “creative” thinking to get the most of it: a challenge when you are pressed for time and resources. Sara Murray took a very different, focussing not on Buddi’s experiences, but on its upcoming applications in self-monitoring and “nudging” to address cravings and so improve health. Given Buddi’s experiences in location tracking for both criminal justice and healthcare applications, this was a little disappointing, but it was also a helpful reminder that “tracking” covers more than just electronic monitoring of location, and opened a line of discussion where many similar issues arise.
    After lunch, there was a session on humane and proportionate tracking of individuals. Anita Dockley of the Howard League for Penal Reform expressed their concerns that electronic monitoring of offenders expands the “carceral space” to the home, and begins to encompass familial and social relationships. By contrast, Tom Sorell of Warwick University expressed the view that of all the intrusions that the state can make into your life, electronic monitoring is relatively mild (compared, say, to tapping your phone, or posting a watch on you). Our very own Amanda Keeling (of Leeds Unversity’s Centre for Disablity Studies) discussed legal issues related to the monitoring and detention of disabled people, including some important rulings that had set out the limit of how people could be monitored or detained for “their own good” on the basis of disabilities. Richard Powley of Age UK discussed the pros and cons of tracking technology particularly for people with dementia, adapting a quote from Bishop Heber to suggest that “every (technology) pleases: only man is vile”. In other words, technology offers much potential for benefit – but also misuse.
    The final session saw Magali Ponsal of the Ministry of Justice and Mark Griffiths of G4S offer perspectives from the civil service and industry on the future of electronic monitoring, noting the incremental nature of changes to the underlying technology since its introduction (a contrast to the radical changes in consumer technology and self-tracking), and the significance of the human and policy systems that went around the technology. This led to some discussions about the business models around electronic monitoring, and the extent to which it is outsourced to the private sector in different countries.

    So, after four workshops, what are the lessons to be learned? I can only speak from my own point of view as an engineer – criminologists, social scientists, and ethicists (among others!) may well have different views – but here are my thoughts:
Technology for locating or recognising someone or something is pretty well-developed, and our capacity for doing it reliably and more efficiently will continue to improve. The drivers for location, computer vision and biometric monitoring in robotics applications will ensure that these developments continue irrespective of their value for the tracking of humans.  On the other hand, I don’t see that any game-changing technologies are likely to fundamentally alter the landscape.

Transdermal monitoring – of blood sugar, alcohol, and so forth – is likely to get more advanced, and become increasingly integrated into worn or carried devices. We’ll see more of this electronic monitoring within healthcare. Whether it’ll ever really take off beyond the quantified self crowd, I don’t know, but I daresay if people can be monitored more easily at home, that will be taken up as part of the wider telehealth drive.

The big issues are less to do with developing the technology itself (which, as noted above, is already pretty good and has plenty of drivers to advance), than about human and sociotechnical issues of tracking. This is true for all forms of tracking, from electronic monitoring of offenders as an alternative to prison, to self-tracking of biometrics. Matters of who owns data (legally); who possesses data (practically); how it is prevented from being accessed by unauthorised parties; and who knows what data is held about them. The risks of “black box” machine learning algorithms facing inadequate critique, yet being used to make crucial judgements and decisions. Matters of consent and coercion, and the wider problem of foreseeing the societal risks of fast-moving technologies before they become widespread.

I don’t have any solutions to these: just that these are issues that we will need to grapple with if we hope to control technology (to paraphrase Geoffrey Vickers) rather than have it control us. Of course, identifying these issues is a necessary first step towards grappling with them: and that, of course, is the purpose of the network. The next step, of course, is to begin grappling with them…

Month in Review: October 2017

If the end of September sees the pre-teaching rush easing off, the end of October sees teaching very much in full swing. There's been a lot going on, a lot of new things starting, but not a lot finished. Unless you count lectures and tutorials and project meetings delivered. I've taken to adding them to my task list, so that every lecture can be ticked off. Otherwise, you do a day of solid teaching and think: "I've got nothing done!". I'd hope the students - whose fees are paying for those tasks - don't see it that way, and makes it no sense to manage your time as if teaching were a drain that got in the way of the real work.

Anyway, I've got three undergraduate team projects for MEng/MDes students, and seven dissertation projects under way. I had industrial visitors in to sponsor an undergraduate project for our level 2 students.

Otherwise, three proposals have been on the go - one died in the internal sift - and I've been hard at work trying to map out the PACLab technology roadmap, so we can keep on top of the tech we need, particularly as VR is becoming a bigger part of our work.

Speaking of which, some of our work with Dubit (particularly driven by Faisal Mushtaq) on health and safety of VR, has now been published, and achieved some note from the media.

Also excitingly, the Apex project I'm doing with Stuart Murray on "Engineering the Imagination" has now been officially announced. We've known about it for a month or two, but had to keep it under wraps! Now we can announce to the world that we'll be doing some critical design of our own, exploring how engineers respond to cultural theory by designing a prosthetic hand to communicate empathy. Really exciting stuff.

November promises to be just as exciting: the grand finale of the Tracking People AHRC network is taking place on the 9th. I'm really looking forward to it: stay tuned!

Present in Absence, pt 2

I've been trying to think a bit more about this whole issue of proximity, and projecting the self that I addressed last month. I feel like there's something in it, but I haven't really figured it out yet, and I may well be retreading old ground. Still, I wanted to dedicate some time to thinking through Virtual Reality, Augmented Reality, Companion Robots and the self, and what better place to think aloud than on the blog? It is, after all, what it's here for. So, buckle in - once more I will be thinking off the top of my head and straying dangerously outside my discipline. Feel free to correct me.

I mooted that there might be five levels of proximity at which something can be found (slightly reworded, you'll note!):

1) Integral: Inside; built into you. Not removable without surgery. By definition, always in contact with you unless drastic action is taken. 

2) Contact: Anything attached to the exterior of the body. Attached, but detachable. Always in contact with you, but removable without permanent damage. 

3) Reachable: Not attached and normally not in contact with you, but easy to bring into contact.  Within arm's reach.

4) Proximity: Not in arm's reach, but within sight or sound. The same room - no interposing barriers.

5) Remote: Not in sight, or sound, barriers prevent interaction except through some third party or device.

Now, one thing that occurs to me is that this refers to spatial proximity. But is that the most relevant form of proximity when "projecting the self"? It sort of makes sense: projecting myself from "here" to "there" inevitably involves a spatial dimension. But it feels like there's a disconnect between integral/worn and reachable/proximity/remote. As if they deal with different types of proximity.

Spatially, at least, they make sense as part of the same scale. From "inside" to "next to" to "can be made next to through greater and greater effort" - reaching out an arm; walking across the room; walking out of the room and if necessary off over the horizon. "Projection" in a sense brings things closer along this scale. Reaching out my arm brings a reachable object into contact; walking across the room brings a proximate object into reach and then into contact. Would ingesting something then be the next level of projection? It doesn't seem quite right.

And yet, remote interfaces sort of follow this pattern - I can project my sense of sight through not just space but potentially (backwards) through time through the use of a camera and a connected display. A microphone and speaker (and amplifier, I'll grant!) will do the same for sound. Thanks to the internet, light and sound from halfway round the world can be projected to my eyes and ears. Arguably, I am the recipient  of the projection (it is projected to me): would we regard this as projecting the senses?

This brings us to the question of immersion. If I watch TV, am I projecting myself or is it (to quote St Vincent) "just like a window?" It brings something distant closer, but it doesn't project "me" there. VR is different, because it cuts out some of the immediate senses. If I put on a VR headset, I now not only receive visual information from a remote (possibly virtual) environment, but I also lose visual information about my immediate environment. Headphones do the same for sound: haptic feedback the same (in theory!) for touch. That, I guess is the projecting element: the extent to which I sacrifice information about my immediate environment for information about a remote environment.  Projecting myself "there" rather than projecting "there" to "here".

So far, this has all discussed projecting senses from one place to another, but what about projecting actions? The partner to telepresence is teleoperation - the ability to perform an action in a remote space. Of course, the microphone and speaker example works in reverse - I can have a webchat with a colleague in Sweden almost as easily as speaking to the in the same room, our voices and images projected across the continent. In teleoperation, though, it feels like we tend to mean the projection of actuation: of force and movement. Of course, remote control has existed for a long time, and the idea of pressing a switch in one place and an action being performed elsewhere is hardly new. 

Based on the ideas discussed by Andrew Wilson et al at the Cognitive Archaeology meet-up, it looks like humans are uniquely well-adapted to throwing things and this was obviously an important step in our development. For an animal with limited natural weapons or defences, the ability to hit something from a distance at which it can't easily hit back is a huge boon, and perhaps the earliest example of telepresence... 

Crossbows and bows caused such concern in the middle ages that they were banned for use against Christians: "29. We prohibit under anathema that murderous art of crossbowmen and archers, which is hateful to God, to be employed against Christians and Catholics from now on." The ability to kill from a distance without putting yourself at close risk was very much the drone strike of its day.  

Anyway, this is a little off the point: I'm just trying to demonstrate that remote action is nothing new. So how does this fit with our model? If we don't want to look at how close something is to us, but how it's proximity is shifted through technology, does the same model still work?

I mean, could we do away with "reachable"? Is reaching just  a way of moving something from "proximity" to "contact"?  Of course, then "proximity" would run out at the limit of reach. Whether something is across the room or across the world makes no difference once it's out of reach. This then raises the question: is walking a form of projection? For me, walking to something just out of reach is trivial; whereas walking to another room is more effort. There again, that effort will increase the further away something is. I can go and put my hand on an object a mile away, it just takes a lot more time and energy. 

This makes me think a few things:

1) That the categories (integral to remote) classify where a device must be for one to operate it, but make less sense in mapping out "projection" of skills. For example, some devices must be implanted to work correctly (a pacemaker is no use in your pocket); some must be in contact (a heart rate monitor; a VR headset); most must be reachable to be useful - contact is required to operate them, but it need not be constant (a kettle, for example - I need to get within arm's reach to turn it on); then we get to voice activation (Alexa, Siri, etc). This is only about projection insofar as it determines how near I need to be to an object to form an assemblage with it.

2) That these will vary from person to person and object to object: how far I "reach" will depend on my arm length; how far my vision extends depends not just on my eyesight but upon what I'm trying to see. I can read a traffic light from 100 metres, but not a novel.

3) I wonder if time might be a better measure of proximity? That is, if we measure the time it would take to form an assemblage with a given object? Hence, an integral or contact object is instantaneously an assemblage: I am automatically assembled with them. For objects not in contact with me, proximity might be measured by the time it takes me to interact with them. For a kettle in arm's reach or Alexa, the time is a second or two: as good as instantaneous. For objects further away, we can either have a binary distinction ("not instantaneous"), or measure the time it takes (five seconds to cross the room; a minute to go next door; an hour to walk two miles away).

4) Perhaps it is the instantaneous/not-instantaneous distinction that is most useful, since this delineates close from remote, and this gap is what projection bridges. Whether my senses or actions, projection means transferring instantaneous senses and actions to somewhere they would not normally be possible, rather than having to take an additional action to get to the relevant place.

Maybe the mapping isn't that useful, then? Or maybe it's useful in mapping the links required to form an assemblage with a device? Perhaps the question is - why am I interested in this? Why do I want to map this out in the first place? I feel like there's something important in here, but I'm not sure.

Let's try a few examples. Ordinarily, if two people wish to engage in conversation, they would need to be in proximity. Hence, in the image below Persons A and B can hold a conversation with each other, but not with Person C (who is remote from them - in another room, building, or country).

Add a telephone (mobile or otherwise) into the mix, however, and as long as it is within reach of both parties (and both have a network, charge, etc.), speech across any distance becomes possible:

Now, there are some complications here. In this example Person B can converse with  Person A (who is in proximity) and Person C (provided both B and C are in contact with their phones). Persons A and C, however, can't converse with each other. Clearly, this need not be the case: what if the telephone is set to speaker? Now, Person C is effectively "proximate" to both Person A and Person B - and no one needs to be physically holding the phone. Of course, Person A and B can see each other, and if anyone gets too far from the phone, their voice will no longer carry, etc. 

A similar issue might be imagined in terms of shaking hands. A and B can shake hands, but only if they are in Reachable distance of each other. Being in earshot of each other isn't sufficient. A telephone won't help A, B or C shake hands with each other, no matter how f

good the speaker and microphones are. 

There's more to this, as well. We need to differentiate different senses and actions. For example, the telephone carries audio information, but not visual information.  Skype or WhatsApp (among other videoconferencing apps) can carry both. They can't carry touch. 

Is it worth thinking of proximity in conductive terms? Conducting information, maybe? Are these assemblages really about "conducting" information? Conducting isn't necessarily the right term - clearly, air conducts sound, wires conduct electricity, but can we speak of electromagnetic radiation being "conducted"?  It hardly matters. For my purpose, these assemblages are analogous to forming a circuit: when the circuit is broken, sensing or actuation breaks down. 

Now maybe that's what I mean by projecting the self - forming sensing or actuating circuits/assemblages with places that otherwise wouldn't be possible: be that because they are remote, or virtual, or just out of reach of our every day capabilities. That's an interesting thought, and worth pondering more: and for that reason, it seems as a good a place as any to end this post.

Month in Review: September 2017

September is always slightly white-knuckle, as the start of term hurtles into view. Lectures need to be ready, Web pages up-to-date, handouts printed, exams prepared - or you're in for a really hard time once teaching starts.

This year, the problem has been compounded by a big research grant getting approved in principle and needing detailed negotiation to ensure that moves to approval in practice; a small research grant in the same sort of state; the launch of the N8 Robotics and Autonomous Systems Student Network down in Sheffield; and chairing a session for LUDI at the AAATE conference (also in Sheffield). All great things (and a nice break from teaching prep!), but all needing to be slotted into a busy time.

Still, it's done: lectures, handouts, exams, ready; modules launched; new and returning tutees welcomed; teaching underway. Which isn't to say that it's an easy ride from here, but the start of term is always a nice point - when you can draw breath, mop your brow and get down to actually teaching instead of just thinking about it. And the benefit of prepping over the summer is that research marches on, rather than coming to an abrupt halt.

Anyway, I promised highlights of the summer, so here they are:

1) Getting MagOne working. Or rather, undergraduate summer interns Jamie Mawhinney and Kieran Burley getting it working for grip and posture applications respectively. Application specific calibration and housings need to be developed, but we've achieved proof of concept for both grip and posture, and the hardware and software for running off an Arduino Nano are in place. Low cost there-axis force sensing, here we come!

2) The fully housed PSATs getting up and running... and getting prepared for prehension studies. Low-Cost market tracking, here we come!

I think that'll do for now. In the meantime... back to teaching!

Present in Absence: Projecting the Self

Last week saw me in Sheffield for the launch of the N8 Robotics and Autonomous Systems Student Network on behalf of Robotics@Leeds (it was great, thanks) and the AAATE conference on behalf of LUDI (also great, thanks!). There are probably blog posts in both, but I mixed in some of my Augmenting the Body/Self duties by catching up with Sheffield Robotics' Tony Prescott and Michael Szollosy, by picking up a MiRo for Stuart Murray to show off on behalf of the project at the Northern Network for Medical Humanities Research Congress, and using my extended commute to spend some time going through our thoughts from the summer workshop. I'll keep those under my hat, since they'll be going into a grant, but between all these things and presentations at AAATE about NAO and ZORA, and with articles appearing about VR at Leeds, I've been thinking a lot about the limits of the body and the self.

So, I thought I would work through them here, at least to get them straight. Bear in mind this is me thinking off the top of my head without a proper literature review, or any kind of philosophical expertise on self  or consciousness. It's a hot take from an engineer responding to the ideas swirling around me: feel free to correct me.

The limits of the body and Deleuzian revisitings of assemblages and rhizomes have been covered elsewhere on this blog. The skin, as I've noted before, is a pretty handy boundary for delimiting self and other, and the one that we probably instinctively default to. Questions of whether my clothes or equipment should be regarded as part of "me" might seem trivial. The answer is obviously no, since I can put these on and off in a way that I couldn't with any other part of me: even hair or nails, while easily shorn, are not rapidly replaced except by extensions or false nails. Yet, if we take ease of removal and replacement as indicative of the boundary of self, then (as Margirit Schildrick pointed out at our Augmenting the Body finale) what about the human microbiome? The  countless bacteria that we lug around inside us? They aren't easily removed or replaced, though they can be - I can swallow antibiotics and probiotics, I guess. Yet, if I have an abscess, it's not easily removed, so is that part of "me"? And by extension, what about a tattoo, or a pacemaker, artificial hip, or insulin pump? I have none of these, so the question is perhaps facetious. But I do have a dental implant: an artificial tooth screwed in to replace the one knocked out on a school playground in the 80s. Is that "me"? Or does it have to be plugged in to my nervous system to be "me"? In which case, my surviving teeth would count, but not the false tooth - what about hair and nails? Do they count? They don't have nerves (do they? I'm getting outside my field here), even if they transmit mechanical signals back to the skin they're attached to. My tooth isn't like my clothing - I can't take it off and put it back in any more easily than my other teeth: it's screwed into my jaw. I can't swap it out for a "party tooth", I don't need to take it out at night. I treat it exactly as I do my real teeth and 99.99% of the time I'm not even conscious of it, despite it offering a block which is absent sensation when I drink something hot or cold.  Which feels really weird when I do stop to think about it, but after nearly three decades, that very rarely happens.

Of course, the answer is probably: "does it matter?" and/or "it depends". Never has the question of whether that tooth is or isn't "me" arisen, and whether our definition of "self" should extend to walking aids, hearing aids, or glasses will almost certainly depend on context. And for my purposes, I'm sure the extent of body and self matters in engineering for at least one reason: telepresence.

Telepresence crops up in three contexts that I can think of: teleoperation (for example, operating a robotic manipulator to clear up a nuclear reactor or robotic surgery such as Da Vinci) - the ability to project skilled movement elsewhere; telepresence robots (to literally be present in absence - sending a tekeoperated robot to attend a meeting on your behalf); and virtual reality (the sense of being somewhere remote - often not physically real).

So this got me thinking about different levels of proximity to the self that technology can exist at. Here's what I thought:

1) Integral: anything physically under the skin or attached to the skeleton; where some form of surgery is required to remove it. Pacemakers, cochlear implants, orthoses, dental implants, insulin pumps. I originally moored "internal", but felt I needed something to differentiate this from devices - camera capsules for example - that are swallowed but only remain inside temporarily.

2) Contact: Anything attached to the exterior of the body: clothes, an apple watch, a fit bit. Also prosthetics. I wonder if puppets fall under this category? Glove puppets at least.

3) Reachable: Anything unattached that I can interact with only if I can get it into contact with me. A mobile phone, maybe - though voice control such as Siri would affect that. Remote control devices likewise - though I would argue that my TV remote requires me to physically touch it. It extends the device, not "me". 

4) Proximity: Siri and Alexa are interesting. Do they extend the device? Or me? I mean, if I speak to another person, I don't regard them as me - but I project my voice to reach them. I extend an auditory and visual (and olefactory!) presence around me. So a camera for gesture control, or a microphone for voice recognition can be activated beyond arm's reach.

5) Remote: At this point, we're talking about things that are no longer in the immediate physical environment: out of sight and hearing. Perhaps another room, perhaps another city, perhaps half way round the world - perhaps in a virtual environment. This is your fundamental telepresence or virtual reality.

In a handy image-based form:

It's not very well thought out: a first iteration rather than a well-founded theory, but I find it helpful in puzzling through a few things. One of these is as a simple way of noting the physical proximity of a piece of technology required for me to form an assemblage with it. An insulin pump or pace maker must be integral; glasses and clothing must be in contact; a smartphone must be reachable; a screen or voice control must be proximate. I can't form an assemblage with anything remote except through an intermediary - for example, I can't talk to someone on the other side of the world, except by getting a phone within reach. 

The other is that we can locate the range of projecting different senses (seeing, feeling, hearing something outside the range of our usual senses) and projecting action (pushing or gripping something we otherwise couldn't; projecting our voice further than usual), and also the relative location of the device in question. For example, a "feature phone" (as in a mobile phone that isn't a smartphone), requires touch - it needs to be in physical contact with me - for me to operate it - but it allows me to hear and speak to someone miles away. Provided, of course, that they also have a phone of some sort.

To what extent am I projecting myself in using such a phone? Probably not much: I don't really regard myself as being in proximity with someone I call. What if I make a video call using Skype or 

Adobe Connection or Face Time? Am I projecting myself then? What if I'm a surgeon using Da Vinci? Am I projecting myself into the patient? There's a whole bunch of questions about what constitutes "self" there, but I feel like the distinction of different types of distance is useful. 

Anyway, I just thought I'd put it up there. It's a work in progress, and I'll have a stab at mapping out some examples to see if it's useful. As always, thoughts are welcome!

Month in Review: August 2017

It has been a busy and (fortunately!) productive summer. The corollary to doing much and getting little done (or at least, little finished!) is that at some point things begin to fall off the To Do list. I'm not at the stage of having everything done, but I'm certainly in the "closing" stage of my summer To Do list. Things that have slowly trooped on are coming to a close: MSc and Summer projects finished; dissertations and resits  marked; a review written; an exam and a conference paper almost finished; handouts printed... a big grant getting dangerously close to signed off; big chunks of analysis software completed; PSAT calibration underway. Still a lot of open tasks, but I'm getting there.

I gave my talk at the Superposition on Humans, Robots and Puppets (with Anzir Boodoo and Samit Chakrabarty): it was a really good event and really got me thinking for the Augmenting the Body project. I finished reviewing Elizabeth R. Petrick's Making Computers Accessible for Disability and Society: a history of the development of accessibility features in personal computers from roughly 1980 to 2000, rather than a "how to" guide. I've written a new marking scheme for a module, and been setting up an industry-linked project for our students.

In September, I'll mostly be gearing up for the start of term: we're less than four weeks away from the start of teaching now; one of those weeks is fresher's week, and another I'm mostly in Sheffield for the N8 Robotics students network, and AAATE 2017 (where I'll be chairing a LUDI session on inclusive play). I want to get my exam written before term starts: I've written three of the five questions. And I need to prepare dissertation topics.

Busy times! I'll try to take stock of exactly what I got done over the summer at the end of next month, as term begins...