Variable reward schedule question

[hunterchristian]

Hi, I have a question regarding moving into a variable reward schedule. My dog is sitting on cue easily. I am still clicking and rewarding every time his butt touches the ground. In a few days I would like to start spacing out the rewards a little and am wondering if I should still click every time he sits, but only reward every couple of times. But then I feel like that would uncharge the clicker. So I am wondering, if a click is always accompanied by a treat, do I refrain from clicking on the variable training schedule as well? Thanks, just confused.

[Adrienne Farricelli]

Yes, a click should always be followed by a treat so you shouldn't do clicks without treats. However, as mentioned in the other post about increasing distance, at this point once your dog performs the behavior on cue, you may want to start fading the use of the clicker if your dog is at the point of fluency and you are starting moving to a variable reward schedule and adding some challenges.

https://www.braintraining4dogs.com/members/brain-training/clicker-training/

 

However, worth precising, it's OK to revert to use of the clicker when you move to working around certain distractions (like training a dog to sit in the yard or further ahead despite a cat is crossing the road) and you want to reaffirm that your dog is still doing a good job despite the distraction. There are also some other exceptions where using the clicker for longer may help (eg. complex behaviors such as chained behaviors).

 

Don't forget that although you aren't rewarding every single sit, it helps letting the dog know he is still doing a good job.

Edited July 23, 2018 by Adrienne Farricelli

[JMFS]

Rather that begin a new thread I thought I'd revive this one and include the link. https://www.braintraining4dogs.com/members/archive/dog-listens-treats/

 

How long would you say before phasing out treats? Gertie seems like one of the stubborn dogs mentioned in the article so I'm guessing I'm being unclear or perhaps I make a motion she's detecting when I have treats and I skip that when I don't. Should it take a matter of days, weeks, or perhaps even longer, before we start the variable schedule? We've been working on sit where I tell her to sit, when she does I give her a treat. She does this unerringly when I have a treat in my hand. Goal is to get her to sit when she hears the word - the squirrel population Boomed this year so I've been focusing on desensitizing that prey drive and I want her to go into a sit when she sees a squirrel (or at the least when I say sit and there is a squirrel present). The cat isn't helping with this issue - he doesn't listen for anything. >

 

Issue: Gertie won't sit when I have my back to her so I'm guessing she needs more reinforcement and does a solid sit without treats before we move up to that step. That's what brought up the question of "about how long" before entering variable. We've been working on this in earnest for 3 days and I'm thinking I'm expecting too much too soon.

 

 

Thank you as always!

[Adrienne Farricelli]

 

How long would you say before phasing out treats? Gertie seems like one of the stubborn dogs mentioned in the article so I'm guessing I'm being unclear or perhaps I make a motion she's detecting when I have treats and I skip that when I don't. Should it take a matter of days, weeks, or perhaps even longer, before we start the variable schedule? We've been working on sit where I tell her to sit, when she does I give her a treat. She does this unerringly when I have a treat in my hand. Goal is to get her to sit when she hears the word - the squirrel population Boomed this year so I've been focusing on desensitizing that prey drive and I want her to go into a sit when she sees a squirrel (or at the least when I say sit and there is a squirrel present). The cat isn't helping with this issue - he doesn't listen for anything. >

 

Issue: Gertie won't sit when I have my back to her so I'm guessing she needs more reinforcement and does a solid sit without treats before we move up to that step. That's what brought up the question of "about how long" before entering variable. We've been working on this in earnest for 3 days and I'm thinking I'm expecting too much too soon.

 

 

Thank you as always!

 

This is a great question! To answer your question, moving from a continuous schedule to an intermittent one is not a clear cut process, but as a general ballpark figure, you should expect to move to a variable schedule once your dog performs the behavior on cue at least 80 percent of the time. However, watch your schedule when exposing your dog to new challenging situations. You may have to briefly go back to continuous and use higher value treats for those occasions.

 

For sake of an example, when your dog learns to sit reliably in your living room (like at least eight times out of ten,) you may start giving treats randomly, but then, once you're out in the yard, where there are more distractions around, your best bet is to move back to a continuous schedule temporarily until your dog responds reliably in spite of those distractions.

 

 

Now, a variable ratio reinforcement schedule entails reinforcing responses only some of the time. Mary Burch and Jon S. Bailey, in the book "How Dogs Learn" compare this to the way slot machines, fishing and the lottery work. This means no reinforcement at all is delivered at times and this can cause frustration. This is why it's important to still provide some form of reinforcement but it doesn't always have to involve food. You can therefore switch between different types of primary reinforcers that dogs are naturally drawn to such as food (chicken, hot dogs, freeze-dried liver) and natural activities the dog perceives as reinforcing (going out the yard to explore and exercise, playing with other dogs, chasing a tossed ball, sniffing a bush).

 

Several ideas of using activities a dog enjoys to reinforce behaviors can be found in the Premack Principle article here: https://www.braintraining4dogs.com/members/archive/premack-principle/

 

Some reinforcement substitutes can also be mixed in and these include several secondary reinforcers most of us are familiar with (praise, pats, belly rubs or even the opportunity to perform another behavior). Such reinforcers though in order to be effective need to have a strong conditioning history consisting of being paired consistently with primary reinforcers before being used on their own and they also need to be occasionally maintained to preserve their reinforcing power. For example, if we say "good boy" a split second before we give a dog a treat, the "good boy" assumes a positive connotation. Now, we can't use "good boy" on its own too many times (and preferable not in a row) or it risks weakening. As a start, a secondary can be used 20 percent of the time without a primary while using it 80 percent with a primary.

 

 

Now, something to be aware of is "ratio strain." When moving from a continuous schedule of reinforcement to an intermittent one, care must be taken to do this gradually. Just picture what often happens when a person is used to the remote predictably turning on the TV at a touch of a button every time, every single day. That day though, when the remote fails to work, (coincidentally right when a big football match is on), watch that person pressing and shaking the remote, pressing harder, and perhaps, in a person with a low threshold for frustration, watch him/her cursing and possibly even tossing the remote against the wall!

 

A classic example of ratio strain in humans: just watch what happens in workplaces across the globe when workers are overworked and underpaid. Rebellion and subsequent strikes take place. In dogs, asking too much and giving a low rate of reinforcement frequency may lead to dogs getting frustrated, showing displacement behaviors and even losing interest, walking away and giving up. Some animals (think killer whales) may even become aggressive.

 

So to prevent this from happening, we must stretch the ratio very gradually. In a dog in the process of being trained to sit, we would therefore, start by giving a treat to the dog for every successful sit at first (CRF, continuous reinforcement), and then, as the dog responds at a steady rate, we can start giving the treat every other sit, then we can start rewarding randomly like the third sit, the second sit, the fifth sit, etc. (while always giving praise or other reinforcement substitutes)

 

This is a good time to start raising criteria, raising the bar and paying attention to what the dog does so we can start picking out only the best sits to reward, so that we can improve quality. Once we have successfully stretched the ratio, we should see a dog who is on his toes and eager to work for that random primary reward, yes, just like a gambler playing the slots at Vegas!

 

To answer your question, there can be several possibilities as to why Gertie seems to act stubborn at times. It could be your body language is revealing something when you are planning on giving a treat and when you are planning on just using a secondary reinforcer like praise and she's picking on that. It might help to make a video of yourself and then watching it to see whether you are giving out some clues.

 

Now, that I re-read your question though, I am not sure at what point you are in the process of training since you mention the stubborn part in article and that is mentioned in the fading the food lure part. Fading the food lure and giving food on a variable schedule are different processes. This therefore raises some questions: How did the process of fading the food lure go? Can you get her to sit without using a food lure? What are you doing to tell her to sit (voice, upward empty hand motion?)

 

About her not sitting with you facing her back, you may need to split this exercise into smaller steps. She likely has grown very dependent on seeing your face and your body facing her and this is normal since this is how we always ask a sit. You can therefore try asking her slightly sideways, then a little more, then more until you are facing her with your back. However, she might not be ready for this yet if you are still in the process of fading the sight of food.

 

Hope this helps, let me know if you need help with the fading the food part and I can suggest more troubleshooting options.

Edited October 29, 2018 by Adrienne Farricelli

[JMFS]

I was thinking variable and fade were interchangeable so I'm a little hazy on that point. To me they seem about the same with the ultimate goal being wanted behavior without constantly rewarding said behavior.

 

I did start paying closer attention to my own body language and noticed she'll sit if I move my right hand forward while it's closed (like I have a treat). Then I give her the treat from my left hand. She ignores the hand signal I initially intended for sitting so I'll just throw that one out and use the closed fake treat hand move since it's working. I must not have been sincere enough in my communication. It's funny - the things I don't give thought to are the things she picks up faster. Instead of "stay" she understands, "I'll be back." I think I sound more genuine, even though it's a sentence, so she latched on to that phrase.

 

Here's our routine for potty breaks since that's when the sit / leave / come are going to be the most important:

 

Sit at the door, I go scan for squirrels, throw a stick in their vicinity to make them go up the trees (I'm still trying to figure out what will make them scatter when they see or hear me - they're oblivious), go back to the porch to tell Gertie to come, have her sit again so I can close the door, then we walk to her potty spot. Nighttime is the easiest because the squirrels are tucked away safe and sound so I've been able to eliminate treats for that outing.

 

Currently I'm rewarding with a treat and "good girl," for each sit. I'll randomly ask her to sit and I reward her with a treat, ear scratches, and praise when she does that because there's a Lot going on when we're in the yard.

 

I do have to give her credit, we've made progress since yesterday. She's less tense now when we go to the yard and there are four squirrels making their twitchy movements, tantalizing her prey senses. I have to stay on top of it, though. Sometimes those dang squirrels are hard to see and I'll tell Gertie "ok" when one of those stinkers is still playing around. She'll interpret that as, "go get it," so I spend a lot of time watching her ears and posture because those are dead giveaways that something interesting is afoot. I was able to get her to sit on the sidewalk with me and watch the squirrels. She eventually went into a down and I rewarded her big time for that one. We're also working on not being an irritant to the man who rides his bicycle by our house. She dd have a relapse due to a brave (or dumb) squirrel who decided to make a run for it right past us. Sit, come, leave it, went unheard. Once the rodent was on the other side of the gate Gertie bounded over to me. I wasn't sure whether to praise her for coming or say nothing. I opted for treat and praise for coming. (Edit: I missed a cue this morning when I was shooing squirrels away. I thought she couldn't see me but she sure the heck did and she dang near climbed the tree - it was impressive, but Not what I was hoping for. Ah well, back to step 1 we go.)

 

I'll be glad when the squirrels have all their goodies gathered and hole up for the winter! There are no less than 8 of them in the front yard at any given time. Momo keeps the back yard fairly clear. Our front yard is full of Disney movie carefree squirrels who might break out into song and dance; our back yard hosts headbanging death metal squirrels - only the toughest guys hang out there. It would not shock me to see one of them sporting piercings and tattoos, or taking swigs out of a flask.

Edited October 29, 2018 by JenniferM.

[Adrienne Farricelli]

Yeah, it can be confusing because the use of food is related to both concepts and in both cases there is some form of fading going on.

 

In fading the food lure, we are removing the food lure's presence in guiding the dog into the position we want. The food lure in this case is a prompt, a visual aid to help the dog perform the behavior. So if we are using food in our hands to guide our dog's head upwards with the nose almost pointing the ceiling so that the rump hits the floor, we are using a food lure as something for the dog to passively follow. Food in this instance (shown as a guide to follow) should be used for very brief periods to prevent over dependence on it.

 

We do just a few reps with the food shown in the upward nose -to- back to the head motion, and next, we try to guide the dog into the sitting position using the same motion as when we were holding food, but this time we are empty handed. There is no food, but the hand gesture is the same. Once the dog sits, we then deliver food using our other hand. The sight of food is now out of the equation. The dog no longer expects to see it protruding from our hands. From what you describe, it seems like you are on the right track. Good job in catching what was preventing her from performing the behavior!

 

On the other hand, when fading giving food on a continuous schedule and moving to a variable schedule, we are no longer giving food as a consequence for every single successful behavior, but only part of the time in an unpredictable manner. By the time we are moving to a variable schedule with the dog performing the behavior reliably, (like 8 times out of 10 when we say sit), quite some time has passed since when we were using food to guide a dog into the desired position. We have long ago therefore faded food used as lures.

 

To be a bit technical, the food lure to guide the dog into a sitting position is an antecedent (something presented before the dog performs the behavior) that then becomes a consequence (when the dog is given the food after he sits) meant to reinforce the sitting behavior. On the other hand, food given on a continuous or intermittent schedule is just a consequence (something given after the dog the dog performs the behavior to reinforce it ).

 

All operant behaviors (things we teach dogs or behaviors dogs learn on their own) are made of antecedents-(things that evoke the behavior such as food lures, cues or hand motions,) then the actual behavior (dog sits) and the consequence (a treat, life reward, conditioned reinforcer, something that the dog has been taught to enjoy through positive associations).

Edited October 31, 2018 by Adrienne Farricelli

[Adrienne Farricelli]

Quoted: "I was able to get her to sit on the sidewalk with me and watch the squirrels. She eventually went into a down and I rewarded her big time for that one. We're also working on not being an irritant to the man who rides his bicycle by our house. She dd have a relapse due to a brave (or dumb) squirrel who decided to make a run for it right past us. Sit, come, leave it, went unheard."

 

That is great progress that she was able to sit with you and watch the squirrels! Kudos to you. She is likely not ready yet for squirrels running right past her though, as that level of closeness takes time and instincts prevail. It might not be too easy to practice in similar real life scenarios, because you might not run into too many squirrels bold enough to do something like that again. Practicing some leave its with a squirrel -like flirt pole in the home and then in the yard may come handy to rehearse such happening in hopes that in the future she may be more responsive to you if she can generalize flirt-pole training to a real squirrel.

[JMFS]

Wonderful clarification! I get it now. I am, indeed, working on the variable because we faded the lure (now that I know what I'm talking about I can sound smart!) a while back. Sister Definitely knows How to sit, she does it Perfectly for her Kong Wobbler every morning and evening. We just need to work on my body language. :D I think I would fail if I were a dog - they're definitely smarter than us in the ways of body language!

 

I'm glad you mentioned the flirt pole because that lets me know I'm on the right track! I cleared a space in our living room so we can work on drop and leave. Eventually I'm guessing I can throw in sit instead of leave somehow so I can get her to stop what she's doing and put that butt to the ground (thinking of traffic or other situations where a sit vs a leave might be detrimental).

 

In other news - daily training, even in small doses, yields better results than sporadic training. It's noticeable. :)