tag:blogger.com,1999:blog-11392667310011194882024-03-12T21:43:56.411-07:00Omri's QA BlogOmri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-1139266731001119488.post-83790288819523367462015-02-04T00:31:00.003-08:002015-02-04T00:31:41.199-08:00My Reversim 2014 session video<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
In Reversim 2014 convention, I have given a session called "<span style="color: #222222; font-family: arial, sans-serif; font-size: 0.9em; letter-spacing: -0.03em;">Alive and kicking - evolving QA to meet future challenges</span>". You can watch it <a href="https://www.youtube.com/watch?v=lcyCX2Hzdnk" target="_blank">here</a>.<br />
<br />
Your feedback is welcome!Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com6tag:blogger.com,1999:blog-1139266731001119488.post-50152366260975621522013-12-26T00:47:00.003-08:002015-02-04T00:18:32.948-08:00Evolving QA to Continuous Deployment<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
<div>
<br /></div>
During the last six years in Sears Israel, we have kept shortening the release cycle. During the first two years we deployed a version every two months (give or take a week or two). Later, sometime in 2010, we opted for a two weeks release cycles. This posed many challenges to all teams involved in the process, but we all adapted and the classic roles of the teams did not change much: Product team wrote the spec, R&D wrote the code, QA tested and approved the quality and Operations team deployed it. When we decided to deploy the code "when the feature is done", things had to change. We could no longer afford to have the QA team standing as a Gate Keeper, approving every version thus slowing the process down. Responsibility for the quality of the features was officially shifted from the QA team to the developers and "QA approval" was no longer a mandatory step in order to deploy to Production.<br />
<div>
<br /></div>
<div>
This shift posed a big challenge to the QA team. If we're not the gate Keepers, who are we? If developers are not <b>required</b> to get QA approval, why would they go through all the hassle of "version to qa-> bugs -> fix -> version to qa" ad nauseam? Looking for answers in the web was not very encouraging, as we quickly found out that <a href="http://www.theinquirer.net/inquirer/news/1720797/facebook-qa-team" target="_blank">Facebook doesn't have a QA team</a> and Google says <a href="http://www.youtube.com/watch?v=X1jWe5rOu3g" target="_blank">Test is Dead</a> so what hope did our little team have? Many people were concerned: mostly QA engineers failing to understand their future role (some probably fearing for their jobs) and developers not too happy about losing their testers and having to dirty their hands with the undignified work of manual testing.</div>
<div>
<br /></div>
<div>
The journey took almost two years. From a team of concerned and doubtful QA engineers and managers we shaped a team of professionals who know exactly why are they needed in the release cycle even if the are not mandatory. The major state of mind shift was from a acting as a mandatory link between R&D and Operations to service providers, catering to the needs of other groups in the organization. We're not the Gate Keepers anymore, but we don't want to be.</div>
<div>
<br /></div>
<div>
We are the <b>testing experts</b>. Not manual tests, mind you (though we will lend a hand if a developer asks nicely) but other, specialized testing forms: performance testing, subjective quality (search results quality, recommendations quality, newsfeed quality, etc.), mobile testing, fault and recovery and more.</div>
<div>
<br /></div>
<div>
We provide <b>Risk Analysis</b> and assist developers and product managers see risks in integration with other interfaces. Over time, our application grew more complex, with a lot of moving parts interacting with each other. Due to the QA team being small, sharing information is relatively easy for us and we invest time and resources keeping our QDOs informed on other components they may not have a day to day experience with. This knowledge allows each QDO to identify risks not only in his domain and feature but also in the interaction of new features with other interfaces and domains.</div>
<div>
<br /></div>
<div>
We are the <b>reporters and intelligence officers</b>. When you deploy a version every few weeks, there's plenty of time to investigate and reflect what went wrong and what went right, what are the current problems and where should go next. This task gets more complex when you deploy a version several times per day. When, exactly, are you supposed to stop and reflect? One of the services our QA team provides is weekly reports in which we connect the dots, so to speak, and tell interesting and important stories reflecting the quality in each domain. Not automatically generated bugs list, mind you, but thoughtful analysis of the current quality in a domain, usage patterns analysis, recommendations on where should we invest development resources and more.</div>
<div>
<br /></div>
<div>
Theses are the main services we provide, but there are numerous others as well: we manage bugs reported by support and internal users, we run usability and functionality analysis, we champion specific bugs that need to be fixed and fall between the cracks, we manage interfaces with other QA teams in the organization and we go over domain specific bugs with the relevant domain stakeholders and assist in prioritizing them.</div>
<div>
<br /></div>
<div>
What about automation, I hear people ask. What about what used to be the holy grail of QA future? While we have some automation capabilities, we believe that since the developers own the quality of their code, they should also be the ones concerned with automating the tests. We assist and have our own automation solutions but the majority of automation in our organization lies on the R&D side and it makes perfect sense to everyone.</div>
<div>
<br /></div>
<div>
The software industry changed dramatically during the last 10 years and the QA needs to change and adapt. We were once the gate keepers but we evolved, offering unique services no other team in the organization can: testing expertise, analysis, connecting the quality dots and providing quality related services to whomever needs them. If your organization faces the same challenges, you had better think in advance where you want the QA team to be and what uniqueness can they bring to the game. As the old saying goes, evolve or die.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com7tag:blogger.com,1999:blog-1139266731001119488.post-40766130087532924142013-10-03T06:44:00.000-07:002015-02-04T00:18:57.379-08:00The character traits of awesome managers<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;"><br /></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;">The call </span><span style="font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi;">- Former manager of you gives you a
ring. He is now in a new company and wants you to leave your current company
and join him. Assuming the position, salary and most other factors are similar
to the position you now hold, what personal/leadership traits of that manger
will make you consider leaving your job and join the new venture?<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Under normal
circumstances, employees will obey you because of basic organizational
hierarchy. However, relying on organizational hierarchy as the sole motivator
will hurt their performance on the long run since m</span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">ost
employees feel bound to their managers, not the organization (“<a href="http://businessjournal.gallup.com/content/523/how-managers-trump-companies.aspx">people<span class="apple-converted-space"><span style="text-decoration: none; text-underline: none;"> </span></span></a><a href="http://businessjournal.gallup.com/content/523/how-managers-trump-companies.aspx">join
organizations but leave their boss</a>”) and Unbound employees are more likely
leave when opportunity arises. Furthermore, m</span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">otivated
employees perform better and inspire others while unmotivated employees just go
through the motions. So, Motivated employees = good, non motivated = not good. I'm glad we got that out of the way.</span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: -.25in; margin-right: 0in; margin-top: 0in;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi;">
<span style="color: #e46c0a; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;">Find your Awesome</span><o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">I wrote this post while trying to identify and highlight personal traits which can assist managers in being
perceived as Awesome by their employees, thus connecting the employees to the
manager and increasing the employees’ motivation and commitment. I started by asked myself the following questions:<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<br /></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l6 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Why
do employees admire their managers?<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l6 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">What
is the source of managers’ power?<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l6 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">What
makes people <b>follow</b>?</span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;"><br /></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="text-indent: -24px;"><span style="font-family: Arial, sans-serif; font-size: medium;">There are many articles about character traits which </span><a href="http://voices.yahoo.com/top-ten-characteristics-great-manager-334355.html" rel="nofollow" style="font-family: Arial, sans-serif; font-size: 18px;" target="_blank">mark great managers</a><span style="font-family: Arial, sans-serif; font-size: medium;"> but upon reading them I felt that these character traits help managers be more successful in their work, they are not traits that will mark them as Awesome by their employees and make people want to follow them. So I compiled my own list of character traits which I believe will make employees admire their managers to the point where they'll take drastic actions such as leaving their work in order to work the that manager.</span></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;"><br /></span>
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;">1. Vis</span><span style="color: #e46c0a; font-family: Arial, sans-serif; font-size: 13.5pt;">ionary and going places</span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l5 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You
have a clear vision and your people believe you’re going to fulfill it.<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l5 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">The
vision exists and is achievable<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l5 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">The
vision is shared with the employees<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in; mso-list: l5 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You
actively strive towards the vision and make visible progress<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Quote: “My manager knows where
she’s going. Following her will take me far.”<o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;">2. People’s Person</span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;"> </span></div>
<div style="margin-bottom: .0001pt; margin: 0in;">
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You care about your
people and they know it. <o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l8 level1 lfo3; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Empowering
your people and make them feel they matter<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l8 level1 lfo3; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Generous
with compliments and gives credit when credit is due<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l8 level1 lfo3; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Promotes
team spirit and atmosphere of collaboration<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l8 level1 lfo3; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Authentic
and trustworthy<o:p></o:p></span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;"><br /></span></div>
<span style="font-family: Arial, sans-serif; font-size: 18px;">Quote:</span><span style="font-family: Arial, sans-serif; font-size: 18px;"> </span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">“I <u>love</u> working for her.”<o:p></o:p></span><br />
<span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;"><br /></span><span style="color: #e46c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-style-textfill-fill-alpha: 100.0%; mso-style-textfill-fill-color: #E46C0A; mso-style-textfill-fill-colortransforms: lumm=75000; mso-style-textfill-fill-themecolor: accent6; mso-themecolor: accent6; mso-themeshade: 191;">3. Source of knowledge<o:p></o:p></span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You are a source of knowledge for your people. Working with you is
a constant learning experience.<o:p></o:p></span><br />
<div style="margin-left: .5in; mso-list: l4 level1 lfo4; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Source of new professional knowledge<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l4 level1 lfo4; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Vast professional experience<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l4 level1 lfo4; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Deep grasp of the profession and the product<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l4 level1 lfo4; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Professional could be Technical, Management, Political, etc.<o:p></o:p></span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;"><br /></span></div>
<span style="font-family: Arial, sans-serif; font-size: 18px;">Quote:</span><span style="font-family: Arial, sans-serif; font-size: 18px;"> </span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">“My manager knows more about this stuff than the rest of us
together.”<o:p></o:p></span><br />
<span style="color: #e36c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-themecolor: accent6; mso-themeshade: 191;"><br /></span><span style="color: #e36c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-theme-font: minor-bidi; mso-hansi-theme-font: minor-bidi; mso-themecolor: accent6; mso-themeshade: 191;">4. The coach<o:p></o:p></span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You help people improve and inspire growth.<o:p></o:p></span><br />
<div style="margin-left: .5in; mso-list: l3 level1 lfo5; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You set high standards, demand high standards and don’t compromise<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l3 level1 lfo5; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You show the way to improvement and enjoy mentoring people<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l3 level1 lfo5; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">You share knowledge willingly and encourage others in the team to
do so as well<o:p></o:p></span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt;"><br /></span></div>
<span style="font-family: Arial, sans-serif; font-size: 18px;">Quote:</span><span style="font-family: Arial, sans-serif; font-size: 18px;"> </span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">“She is tough, but she is the best teacher I’ve ever had and she
will help me grow.”</span><br />
<span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;"><br /></span>
<span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">The character traits which employees admire in their managers and
will make them follow those managers are the ones which bestow <u>confidence</u>
in the employees and the first thing people look for in their managers is the confidence they project. </span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">Different people look for different things in their managers but
the one character trait all employees seek in their managers is confidence in
their ability to lead.</span><br />
<span style="font-family: Symbol; font-size: 13.5pt; text-indent: -0.25in;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"><br /></span></span>
<span style="text-indent: -0.25in;"><span style="font-family: Arial, sans-serif; font-size: medium;">When I presented this to the managers in our company, some people asked "</span></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">Where are the other traits such as Honesty, Passion, Communication,
Humor, Commitment, Personal Example, Coolness under fire etc’?" My answer was that w</span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">hile these traits are crucial in order to be a good leader, they
rarely make an employee connect with his manager based on these traits alone,
i.e. the fact that a manager sets great Personal Example does not make her a
leader to follow on the long run.</span><br />
<span style="font-family: Symbol; font-size: 13.5pt; text-indent: -0.25in;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"><br /></span></span>
<span style="font-family: Symbol; font-size: 13.5pt; text-indent: -0.25in;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span dir="LTR" style="text-indent: -0.25in;"></span><span style="text-indent: -0.25in;"><span style="font-family: Arial, sans-serif; font-size: medium;">Different people look for different traits in their leaders. One
person will tolerate an uncaring manager if that manager sets high standards,
another will not consider working with a manager who doesn't know his spouse’s
name.</span><span style="font-size: 9px;"> </span></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">Different leaders are awesome due to different personality traits.
Find your awesome and focus on it. </span><span style="font-family: Arial, sans-serif; font-size: 13.5pt; text-indent: -0.25in;">Remember: he who wishes to be strong on all fronts will ultimately
be weak on all fronts. Focus on your strengths instead of trying to strengthen
your weak points.</span><br />
<span style="color: #e36c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-font-family: "Times New Roman"; mso-hansi-theme-font: minor-bidi; mso-themecolor: accent6; mso-themeshade: 191;"><br /></span>
<span style="color: #e36c0a; font-family: "Arial","sans-serif"; font-size: 13.5pt; mso-ascii-theme-font: minor-bidi; mso-bidi-font-family: "Times New Roman"; mso-hansi-theme-font: minor-bidi; mso-themecolor: accent6; mso-themeshade: 191;">Discuss<o:p></o:p></span><br />
<div style="margin-left: .5in; mso-list: l7 level1 lfo7; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Are there more awesome leadership traits not covered?<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l7 level1 lfo7; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">What are <b>your</b> strong leadership traits?<o:p></o:p></span></div>
<div style="margin-left: .5in; mso-list: l7 level1 lfo7; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 13.5pt;">·<span style="font-family: 'Times New Roman'; font-size: 7pt;">
</span></span><!--[endif]--><span dir="LTR"></span><span style="font-family: Arial, sans-serif; font-size: 13.5pt;">Can leadership traits be strengthened? How?<o:p></o:p></span></div>
<br />
<div style="margin-bottom: .0001pt; margin: 0in;">
</div>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com7tag:blogger.com,1999:blog-1139266731001119488.post-90316611829816475662011-10-18T14:20:00.000-07:002015-02-04T00:19:06.766-08:00Evaluating workers, skills Vs traits<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
How do we evaluate our employees? We know the traits we value in employees, but which traits are more important and which are less? Through gut feeling we may feel that trait A is more important than trait B, but how does each of them translate into day to day performance?<br />
<br />
In this post I will discuss a rather cool method we have developed for determining which traits are the most important for an employee to posses, how to evaluate employees and more. The first steps are rather obvious, but starting from the third step it's getting rather cool. Trust me.<br />
<br />
Some time ago, I had a conversation with one of my Team Leaders, regarding the needed skills and traits for a QFO (QA Feature Owner). We made a list of the traits we valued the most and wrote them on a whiteboard but then the calender reared its ugly head and we were forced to stop and vowed to resume the session "soon".<br />
<br />
"Soon" turned out to be some six months later. When we met again, were no longer two people in the room but six, as I asked other leads from my team and two HR to join the party. After a long discussion we decided that our way for evaluating our employees will consist on the following phases:<br />
<br />
1. At first we decided what are the needed <span style="font-weight: bold;">skills</span> the QFOs needed. The skills we decided our QFOs need are:<br />
<ul>
<li>Task planning</li>
<li>Task duration assessment</li>
<li>Synching other interfaces</li>
<li>Reporting</li>
<li>Following up on features</li>
<li>Context switching</li>
<li>Task prioritization</li>
<li>Problem solving</li>
<li>Specification document analysis</li>
<li>Writing organized test documents</li>
<li>Effective test coverage</li>
<li>Good test prioritization</li>
<li>Bug hunter</li>
<li>Organized bugs</li>
<li>Through bug analysis</li>
<li>Bug prioritization</li>
<li>Etc' (this post is not about the skills required to be a good QFO)</li>
</ul>
<br />
2. After we were happy with the <span style="font-weight: bold;">skills</span> needed, we set about writing down the <span style="font-weight: bold;">traits </span>we thought good QFOs should posses. The list we came up with was something like this:<br />
<ul>
<li>Fire and forget</li>
<li>Broad vision</li>
<li>Self driven</li>
<li>Self improving</li>
<li>Passionate about the product</li>
<li>passionate about QA</li>
<li>Technical skills</li>
<li>Organized and thorough</li>
<li>Businesslike (practical, focuses on getting things done instead of ego games)</li>
<li>Critical thinker</li>
<li>Creative</li>
<li><br /></li>
</ul>
So now we have two lists, skills and traits. Now we put them in an excel table, with traits being columns and skills being rows:<br />
<img alt="" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABCMAAAFKCAIAAACo7gKqAAAgAElEQVR4nO2dT6slW5rWz+fIL3BzlrMzvHDhQOYgB/kJnAnSM3EmjsSR9tAjjkqcC85qrKiIiNXZoCYiIk2TtLW7quk6dlXZ1KAcHN25bsRaK1b8eSPe931+wcNl3zixVzy/Z7/7ZDwZ+9798Lf+8F8ihBC6Vn/wj/7FXZebQQghdLnKPxfi6uHl+fF3//WfIzQu5ZlJyR4aKrR5wIkrgfxn698hmFJQadhfnh9/N7A9vDw//vV//mcIjUt5ZlKyh4YKbR5w4kog/9n6dwimFFQa9pfnx/8zsD28PD/+7o//KULjUp6ZlOyhoUKbB5y4Esh/tv4dgikFlYb95fnxNwPbw8vz41//0T9BaFzKM5OSPTRUaPOAE1cC+c/Wv0MwpaDSsL88P/7VwPbw8vz42z/+QzTXdz98f7kHn7pqZr774ftXKbCfCWsEdUcwpbh8Gq8aSIe/ui/PJFZcI0le7uTkbCfIRyVw/qt/yWt3JuZpgBe+c0vGBC+oBcLL8+NfvPxmUQ8vz4+//qN/iOb67ofvL/fgU1fNzP0VufClOYe9BDwB1gjqHIrLp/EqOfzVfXkmseIaidFzpBbZTniPwj//1b/khTsT8zTAC9+5l19yHMtuQfHy/PiLv/yrRT28PD/+9j/9AzTRdz98f/8nmuiqmbm/HBe+LuewnwxoBDWhMIK6fBqvksNf3ZdnEiuucBmekK3RL43zX/1LXsczMU8DvPCde/l14LHsFiAvz4+3v3hZ1MPL8+Ov/+PfRxN998P393+iia6amfKzGbnZTwY0gppQGEHJTqPDX92XZxIrrlaGl3u4MNtyhA4cpPNf/UtexzMxTwO88J17+a+yY9ktWF6eH//XL/5yUQ8vz4+//Q9/D5X67ofvq4/Rq66aGQ+vyznsJ9MZQU0ojKA8TOMlcvir+/JMYsXVz/Dbf6pxtaUzszX6pXH+q3/JC3cm5mmAF75zXxkvfA8ey24B8vL8+PXnv1zUw8vz46///d9FpaZ/p3K1H2+6ambK1+Kq1+Uc9pPpjKAmFEZQHqbxEjn81X15JrHiGsnQc6QW2Rrhn//qX/LCnYl5GuCF79w7Y47rDQuKl+fHP/2zP1/Uw8vz42//3d9Bd333w/eLe8R11cyUL8RVL8o57CeTGkGdQ+FhGi+Rw1/dl2cSK66RGD1HapHthPco/PNf/UteuDMxTwO88J17Z8xxvWFB8fL8+Cdff76oh5fnx1//m7+N7vruh+8X94jrqpn50Y2m7OxnkhpBnfN6yU6jw1/dnn9VOoyrn+S1v+iuynaCfFQC57/6l7x252CePJkXvnNLzAQvqMWfWS/Pj//9T/98UQ8vz4+/+Vd/gNC4lGcmJXtoqNDmASeuBPKfrX+HYEpBpWF/eX78b3/y80U9vDw//urf/k2ExqU8MynZQ0OFNg84cSWQ/2z9OwRTCioN+8vz45f/+WeLenh5fnz5138DoRVSnpmU7KGhQpsHnLgSyH+2/h2CKQWVhv358b/8j6+Lenh5fkQIIXSt/vc//qbLzSCEEEKL+slPf7aoh5fnxwc2tjXb73//+6stXLalZA8NFdr8nk0WfNtGXHab/2z9OzxkS4mZEmpw88/+6vD3S9v/axqLx7Gx3TflgUnJHhoqtPk9myz4to247Db/2fp3eMiWEjMl1ODmnH1L07j94tcIjejp/afLPcAOVALzpX7+y1/dJQV+johLOVv/DsGUgkrATtNAtvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsNM0kK3K6S8/sXffc5qT6rlKM+ewnwA1fsAGP5NfZyNQkwP6T2m9TIe/IkeZP3OG76ejaZjKLq6Tp+V8J/Nf8pMz3rOdfIbbwvlaxonD3Drqjye7kS5Xbj3uQPW5Wotsm5m1B+z8g3sn+/6XY61oGshWk19nFzpp/clnd5Yq+7EnPT/SDVCDf0icQGRt/jTtaRrbLu/Of7EuVIhrzZ3hWzy9X8Kr1/EbbFhf2rau2Kz/vDg5hxP+eDIKZ7BpjP/qPvxV2P+nxs6mcdQfW0fNA00D2arfNCbvh/K3ef9vI+Y/fX08f0rrz4n5/uqa1QNaT5kcsPirfNzAONQ8kOoTq6/ISKqL1bG1SMf2Yrat9MZt25nvnHfk1exQd17Q147x8PDw+s/W67j4qlVDuER+nMzj6rxqreQ7r+/i8beByensqWa7E6Fqr/rc1r/en7jYNBZdtSJq7Wwt0iLt/JaoxnuVdpoZ+eOp9XgyeP1XYeQ1HVy573YyXf3RKhep7q8+pTUz1acsjmsntA5ja884e3+pxWRaPicLlqJpIFuNf3pq5PFck0Gf7+yss3jG/s7Fp4xcr4wjj0BVf3/tTGBy3v6FwniM2541CNiasRPMd+Zk1fotutd/3pvG4On6TaM6nOX+1p6R4yexVE/X2TMP/AStKmatNBb3rBrd/ukWTQ4+fdvO1qmrk7DzPdiKqP9e6McyOX6kafQHeD7PZQKdF671Tpnj7393bPvjaXGkB1+F/W+W6k9X/c3mqsetpRbfCCOZtA5Y9XLsZ++v39m5uCBNA9lq8/T3f5/Ofzr4lhg8+8jOqsmRd/6ivVYU41CLv9E2JzD/YMniCzf+e208nPFXofWKmJpvDcyGGai+oOU9jZHTjV86V58+sqdlvrPgqhzO1Ehc1VGZv7/2v+M6p6v+tJXzyGBv29l6sRZHsfqUvtX9zhczGWwaazOpEg2+AQeJVmnbH0/bRroadfWlH1y5NTbWTWNxOMczGQmtderqOhs+PzI5S3/9+eP+S1yKpoFsdWDP7h+56v08ePb+r8LFXzH7m8ZmqM7vwfHfI9Wd1Y+wb/u1uyecbWc8zXx12T0zMPnXTtOYvMRz8PE/Y/rzv4q96rAzilWTp2nzO3cxtwPf8ouTaXqixffC5OD7Nn5PY+SYzRF1jj+kaSxO+IirwUW26aghXzXS81d288rVn5o2jW1vhFXZTg5YfLnLndvY9/gceYlfRdNAtkrTNPpPr+68pGmsOmCDgf6ne3f6GcTcnMBp5hfXH38Jqo9H7mmU2vyH0J5Lgf07r9Kei7D+nksSO+pErZe+ZWNyOTLPtvWUkWV3RtQ5fm3TmCSz2eHIa3rg22R8yOe/BEZC7o9Kh2Jw5epPFz8mupjktpkZfJnWhrZqHraxb/M5smApmgay1c5LnOrvuOpPO2/d1jqtM646oLVzzr5h/f4pOuH0o9iWwByqhN3w0mx+1uYzWpjvxzV5sHkGJo9H7mm0wOcUfcbOnnH2bTuv0sj/qqs6KvMXd+3L3XncWr/vc9Ht4AQ+1K5XqrMxP1Er2+pr3SGteujk1kdrWd18T6P/Io7vbC27eN5VGh/yieH5SzPyuBV+69UZWXn+9P7/6bWf9ny6xmem+krNF1x0Mj9gMcbqC9pnb73inZ3VFRYXvIumgWwV4v8XOaLOL6n07GmgrjU/ODYj6vxfbquDOv4/3Zr/mTo/prVU/4DxP9JaB5wmP/9v7tCqvnz+f4GM/F9uRyZ//oaaP33x/TW4cz9mDqWESsBO00C28jz9q7Th93sa9jRQIk1jPqihX7XzRdM4J1ufWuuw00A8y/8LAVQadpoGspXn6YddDSq0+VJ8R7ipiEs52w1N48L7b4lfCKDSsNM0kK08Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgE7TQPZyvP0w64GFdp8KZqGqYhLOVv/DsGUgkrATtNAtvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsNM0kK08Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgE7TQPZyvP0w64GFdp8KZqGqYhLOVv/DsGUgkrATtNAtvI8/bCrQYU2X+rwpmH9VQB81QCKku1+hyGm3f8LAVQadpoGspXn6YddDSq0+VLemkb16UG/O3kkLpQ4W5pGXKWESsBO00C28jz9sKtBhTZfak/TqH6lcet7jqtHTh6PLFge2V/fg9LMiUP5z3bkzXLrDrO3eQ76QgCVhp2mgWzlefphV4MKbb7UUfc07pdE1S6xbef8p7cfX5b1l/KgNHPiUP6zHXmzBBrmuC8EUGnYaRrIVp6nH3Y1qNDmS+1sGvO/fD2haVSf5fOvgdPMiUP5z/bAN4tn+X8hgErDTtNAtvI8/bCrQYU2X2rnp6fmj69qGpcnuRgXUsv2wDeLZ/l/IYBKw07TQLbyPP2wq0GFNl/KQ9No/S1v/0St9V0pzZw4lP9saRpxlRIqATtNA9nK8/TDrgYV2nwpi09PVT/F1Nl5a/+XGItHdpx4UJo5cSj/2W779FT1SM/y/0IAlYadpoFs5Xn6YVeDCm2+FN/cZyriUs7Wv0MwpaASsNM0kK08Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH76qbxeujT+08IIYSu0YeP33S5GYQQQqihe8F49/bN5y9f++KeBlqtJ8c9G3Y1qNDmS3FPw1TEpZytf4dgSkElYOfTU8hWnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAJ2mgaylefph10NKrT5UjQNUxGXcrb+HYIpBZWAnaaBbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYN/YNB5+vK06pfXx5yyFBuV5+mFXgwptvhRNw1TEpZytf4dgSkElYN91T2PbhfuFTQOdL8/TD7saVGjzpWgapiIu5Wz9OwRTCioB+5FNo3WXY7Jz/qC/wuRx64DxnZ0DOhRomzxPP+xqUKHNl6JpmIq4lLP17xBMKagE7Fb3NPp1ovXcaitonaU8oFpj+t2mfwA14yh5nn7Y1aBCmy9F0zAVcSln698hmFJQCdgPbhrzGwKtmwZVNyNNo3qK/rP6O6sH0DSOkufph10NKrR5wIkrgfxn698hmFJQCdgP/vRU9fGtdgNh2z2NnaVicalbtwihDfI8/bCrQYU2DzhxJZD/bP07BFMKKgH7SU3jNrvc99k06BiHy/P0w64GFdo84MSVQP6z9e8QTCmoBOwnfXpq8LJ+/vT5YaafnqoaRnvkefphV4MKbR5w4kog/9n6dwimFFQC9l1N43Bdfn3fvy2DNsjz9MOuBhXaPODElUD+s/XvEEwpqATsNI2pAe5pHCvP0w+7GlRo84ATVwL5z9a/QzCloBKw+2oaKJ88Tz/salChzQNOXAnkP1v/DsGUgkrATtNAtvI8/bCrQYU2DzhxJZD/bP07BFMKKgH76qbxeujT+08IIYQQQggh1NK9YLx7++bzl699cU8DrdaT454NuxpUaPOAE1cC+c/Wv0MwpaASsPPpKWQrz9MPuxpUaPOAE1cC+c/Wv0MwpaASsNM0kK08Tz/salChzQNOXAnkP1v/DsGUgkrATtNAtvI8/bCrQYU2DzhxJZD/bP07BFMKKgH7xqbx8ONt1SkHjz/wGy34cowL5Xn6YVeDCm0ecOJKIP/Z+ncIphRUAvZd9zS2XcGf0zRoF05C8Dz9sKtBhTYPOHElkP9s/TsEUwoqAfuRTaN1l2Oyc/6gv0h5WLnzfkz1KZMDOn7uB7TuzwxyDe4ZOWziZ3CFyZEt2yfL8/TDrgYV2jzgxJVA/rP17xBMKagE7Fb3NKo9odzTuqbvr9BZduRZndUWVx7kWoXT2TM/fjGQ/pFXyfP0w64GFdo84MSVQP6z9e8QTCmoBOwHN43536O3/j6+6mZz06iet7/s4in2cz38eFv1xI7J1rKLKV0iz9MPuxpUaPOAE1cC+c/Wv0MwpaASsB/86anq41vtb9xbF/Tzx6u6gUXTGORqkVY1/sRq09gQ3VXyPP2wq0GFNg84cSWQ/2z9OwRTCioB+0lN4za7/N1wubx4LV591p6D1/60s2wfdsQPTSOcUrKHhgptHnDiSiD/2fp3CKYUVAL2kz49NX4dPF/hVnSM1nMHn7VoadxV63SLxkaeuBhXddnW06tEp8nz9MOuBhXaPODElUD+s/XvEEwpqATsu5oGQovyPP2wq0GFNg84cSWQ/2z9OwRTCioBO00D2crz9MOuBhXaPODElUD+s/XvEEwpqATsNA1kK8/TD7saVGjzgBNXAvnP1r9DMKWgErDTNJCtPE8/7GpQoc0DTlwJ5D9b/w7BlIJKwL66abwe+vT+E0IIIYQQQgi1dC8Y796++fzla1/c00Cr9eS4Z8OuBhXaPODElUD+s/XvEEwpqATsfHoK2crz9MOuBhXaPODElUD+s/XvEEwpqATsNA1kK8/TD7saVGjzgBNXAvnP1r9DMKWgErDTNJCtPE8/7GpQoc0DTlwJ5D9b/w7BlIJKwL6raez//unWF3LbAVsY3rngtd/hbS3P0w+7GlRo84ATVwL5z9a/QzCloBKwb28a9+vjPRfK51xkl2exPuOq9XN3jFd5nn7Y1aBCmwecuBLIf7b+HYIpBZWA/YCm8ar51fz9n/Mj7zurx1SXai0yv1if7J8cOeKqtWfR29xVZ9m5sdYZ+wu2djqR5+mHXQ0qtPlSP//lr+6SAj9HxKWcrX+HYEpBJWDf++mp+0KLl+Dzw0aO6R8wv7auWhpfsL/45qVGDqieZbHndBZxIs/TD7saVGjzpWgapiIu5Wz9OwRTCioB+66mcWvfyqj+6Lb+0rl/wLamUT1gfkthpGksnqu/bOvWRL+YzY/xXDY8Tz/salChzZeiaZiKuJSz9e8QTCmoBOx7m8ZtdpF9QtO4tS+vd1aX1lIjfvoHjK8wGNd8WZ99w/P0w64GFdp8KZqGqYhLOVv/DsGUgkrAvr1prC0G850jx4wXg8MXjNU0Fn96lTxPP+xqUKHNl6JpmIq4lLP17xBMKagE7Lvuacz/Hn38dsTkBshtTTGYLDJR9Uf9M5bPGq8fI/db+stWTU7Mz5/e2nP5MFXlefphV4MKbb4UTcNUxKWcrX+HYEpBJWA/4NNTd512sbt43wD5kefph10NKrT5UjQNUxGXcrb+HYIpBZWAPWrTcP4X+eguz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH7kU0Dobk8Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH76qbxeujT+08IIYSu0YeP33S5GYQQQqihe8F49/bN5y9f++KeBlqtJ8c9G3Y1qNDmS3FPw1TEpZytf4dgSkElYOfTU8hWnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAJ2mgaylefph10NKrT5UjQNUxGXcrb+HYIpBZWAnaaBbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYN/eNM78Ugu+NyOuPE8/7GpQoc2XommYiriUs/XvEEwpqATsu5rGfRWLJkC7yCHP0w+7GlRo86VoGqYiLuVs/TsEUwoqAfvxTWN+o6N196N/ZPnT+bk6q1FRXMnz9MOuBhXafCmahqmISzlb/w7BlIJKwH5w07DYOX88P766AvIgz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH7rqaxeFPivrM8YO3Okcc0DbfyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErAfcE+jVQOqO42axq32eSrkQZ6nH3Y1qNDmS9E0TEVcytn6dwimFFQCdsNPT81/2jrywHsayJs8Tz/salChzZeiaZiKuJSz9e8QTCmoBOwHN41b7VNVrf9Wu/rhq/kB/XN1VkMe5Hn6YVeDCm2+FE3DVMSlnK1/h2BKQSVg3940xnXC1f/iJ7jQVfI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsOdpGtzT8CnP0w+7GlRo86VoGqYiLuVs/TsEUwoqAfsZTQMpy/P0w64GFdp8KZqGqYhLOVv/DsGUgkrATtNAtvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsK9uGq+HPr3/hBBC6Bp9+PhNl5tBCCGEGroXjHdv33z+8rUv7mmg1Xpy3LNhV4MKbb4U9zRMRVzK2fp3CKYUVAJ2Pj2FbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYKdpIFt5nn7Y1aBCmy9F0zAVcSln698hmFJQCdhpGshWnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAL27U1j8CssjvpOvRBflMEXeszlefphV4MKbb4UTcNUxKWcrX+HYEpBJWDf1TRGTqDz7d3b6HJncvM9/bCrQYU2X4qmYSriUs7Wv0MwpaASsB/cNOZ3OTpNY3Jw/1nVPa0F5/vnjxfvybRYqk+Zr1Z9euuYFmM1pQ745fM0l+fph10NKrT5UjQNUxGXcrb+HYIpBZWA/chPT7UulOc7q49X/XS+fvWMG4z1n7L2LNUjV+Wz6gCH8jz9sKtBhTZfiqZhKuJSzta/QzCloBKwH3lPY2fTuP24VEwKxvgFt2nTGH/KvIlN9q/lai3r9m7GqzxPP+xqUKHNl6JpmIq4lLP17xBMKagE7Hmaxq19zX1J09hpY/Cki+yXy/P0w64GFdp8KZqGqYhLOVv/DsGUgkrA7qtp7Hncvwq/PwjRNMZ9ttb3I8/TD7saVGjzpWgapiIu5Wz9OwRTCioB+5FN41Z8vKfcU31cPbhz/Pg9jY6x2+xqvnMrYBVLh67qcL6z47O/bJ/icnmeftjVoEKbL0XTMBVxKWfr3yGYUlAJ2Lc3DW8a/7v/8WPQfnmeftjVoEKbL0XTMBVxKWfr3yGYUlAJ2FM1jVV/u0/TOEeepx92NajQ5kvRNExFXMrZ+ncIphRUAvY8TQP5lOfph10NKrT5UjQNUxGXcrb+HYIpBZWAnaaBbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYF/dNF4PfXr/CSGE0DX68PGbLjeDEEIINXQvGO/evvn85Wtf3NNAq/XkuGfDrgYV2nwp7mmYiriUs/XvEEwpqATsfHoK2crz9MOuBhXafCmahqmISzlb/w7BlIJKwE7TQLbyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErDTNJCtPE8/7GpQoc2XommYiriUs/XvEEwpqATsu5qG82+nLrXnSzZCALqV5+mHXQ0qtPlSNA1TEZdytv4dgikFlYB9e9NY+53cpayP37wapeJweZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYj2kad83vcrw+LnfOv8x7/q/zBas3T8r9reYz+Wn/7HO3rdUmZjomxeV5+mFXgwptvhRNw1TEpZytf4dgSkElYD+yaVSvzqsVon8df6vVg7n1+bV+5/HiGcfddg6gZszlefphV4MKbb4UTcNUxKWcrX+HYEpBJWA/o2ks7qzeEOj0gfKJcwPzf3ZWG6krg72o5VNcnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAJ2F02j6mzkmFvtJsP5TWNiA5XyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErAf0zQ6n1Da1jRGPj1VXWHP47VuuZUxIs/TD7saVGjzpWgapiIu5Wz9OwRTCioB+/amcWv899+DH4Ka3Ison9i6mp+3gtYHqPqPW2fvnHoQh8oxl+fph10NKrT5UjQNUxGXcrb+HYIpBZWAfVfTQK8a/KCXpjxPP+xqUKHNl6JpmIq4lLP17xBMKagE7DSNAzS5J4NKeZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYaRrIVp6nH3Y1qNDmS9E0TEVcytn6dwimFFQCdpoGspXn6YddDSq0+VI0DVMRl3K2/h2CKQWVgH1103g99On9J4QQQtfow8dvutwMQggh1NC9YLx7++bzl699cU8DrdaT454NuxpUaPOluKdhKuJSzta/QzCloBKw8+kpZCvP0w+7GlRo86VoGqYiLuVs/TsEUwoqATtNA9nK8/TDrgYV2nwpmoapiEs5W/8OwZSCSsBO00C28jz9sKtBhTZfiqZhKuJSzta/QzCloBKwb2waDz/eVp1yz5dOrHpu6+DTDKCb7+mHXQ0qtPlSNA1TEZdytv4dgikFlYB91z2NbZfdl1ys0y6ukufph10NKrT5UjQNUxGXcrb+HYIpBZWA/cim0brLMdk5f9BfodzTX7l18GTZ1vFVA53ntgzc91/+6nqQ5+mHXQ0qtPlSNA1TEZdytv4dgikFlYDd6p5Gv060nju/ZG/tPOTgzp49BjrsgvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsB/cNFr3BObHVN1c0jRatymONSArz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH7wZ+eqj6+1f6m3889jWPXpGlM5Hn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgP6lp3GbX4k6axggCTWOPPE8/7GpQoc2XommYiriUs/XvEEwpqATsJ316avGivPX021IxWHvw/Jj5Cq2dOw1oyvP0w64GFdp8KZqGqYhLOVv/DsGUgkrAvqtpHC4uzfPJ8/TDrgYV2nwpmoapiEs5W/8OwZSCSsBO00C28jz9sKtBhTZfiqZhKuJSzta/QzCloBKw+2oaKJ88Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH76qbxeujT+08IIYSu0YeP33S5GYQQQqihe8F49/bN5y9f++KeBlqtJ8c9G3Y1qNDmS3FPw1TEpZytf4dgSkElYOfTU8hWnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAJ2mgaylefph10NKrT5UjQNUxGXcrb+HYIpBZWAnaaBbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYN/eNKrfpd1X9ciTv0Ojczq+zcNCnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAL2XU3jvsrgNbrzpoEs5Hn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgP7JpTG5xzO94VG+DTC79Wwfcd3buonTOOD9d9RStk4Ivu3oAABhzSURBVPYRUEeepx92NajQ5kvRNExFXMrZ+ncIphRUAvaDm8Zkz+Tx+M7JAfOD5xf6/T39Far+O0+hZozL8/TDrgYV2nwpmoapiEs5W/8OwZSCSsC+q2nMb2Lc193cNAaXrfaK+f2HEQ8b/NM0xuV5+mFXgwptvhRNw1TEpZytf4dgSkElYD/mnsarjr2nMb7U5OC1pxtvGrdan0F9eZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYszWNVacb8Tx4UtSS5+mHXQ0qtPlSNA1TEZdytv4dgikFlYDdqmncdv8X4ds+PTXioeV5/J4GlWNcnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAL27U1DUIu3X9BcnqcfdjWo0OZL0TRMRVzK2fp3CKYUVAJ2msYKVe/JoL48Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgE7TQPZyvP0w64GFdp8KZqGqYhLOVv/DsGUgkrAvrppvB769P4TQgiha/Th4zddbgYhhBBq6F4w3r198/nL1764p4FW68lxz4ZdDSq0+VLc0zAVcSln698hmFJQCdj59BSylefph10NKrT5UjQNUxGXcrb+HYIpBZWAnaaBbOV5+mFXgwptvhRNw1TEpZytf4dgSkElYKdpIFt5nn7Y1aBCmy9F0zAVcSln698hmFJQCdh3NY2jvlmi+sXe8/171jzQXjKtotsQhefpt1ZK9tBQoc2XommYiriUs/XvEEwpqATs25vG/aLz2AtxJ+3ihGWdOLGm8zz91krJHhoqtPlSNA1TEZdytv4dgikFlYD9gKZxV+srtBd3lqWlur/6lMmy48+977l1b8tUccZZbrMyNjG/1t5ivIt0nSM7ziceFl/ZiTxPv7VSsoeGCm2+FE3DVMSlnK1/h2BKQSVg3/vpqXKtVdevi0/sPJ6of66Oh5afw1mq5xq5xO+fqx9F/1kjNgZj6cvz9FsrJXtoqNDmS9E0TEVcytn6dwimFFQC9l1N4zZ8Qb/4V+OrHs9X3rDO4uKDT9lftxYjWmwaq6JYPLJaV1pLLU6Y5+m3Vkr20FChzZeiaZiKuJSz9e8QTCmoBOx7m8Zt5YV4deeGprGqPPhvGv2Idh65duecqGNgsW94nn5rpWQPDRXafCmahqmISzlb/w7BlIJKwL69afSvmGka4zco/DSNxVOMvJoTeZ5+a6VkDw0V2nwpmoapiEs5W/8OwZSCSsC+657G5O+zH4qtPEd1f7lnvF1MLt9byy4+t3+iQdt9Y7eV1/3zRVpW527XPr11ZOvgwZ9W5Xn6rZWSPTRUaPOlaBqmIi7lbP07BFMKKgH7AZ+eumvxohM5l8Ur6Hn6rZWSPTRUaPOlaBqmIi7lbP07BFMKKgE7TQN9E00D9txQoc2XommYiriUs/XvEEwpqATsRzYNhObyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErDTNJCtPE8/7GpQoc2XommYiriUs/XvEEwpqATsq5vG66FP7z8hhBC6Rh8+ftPlZhBCCKGG7gXj3ds3n7987Yt7Gmi1nhz3bNjVoEKbL8U9DVMRl3K2/h2CKQWVgJ1PTyFbeZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYaRrIVp6nH3Y1qNDmS9E0TEVcytn6dwimFFQCdpoGspXn6YddDSq0+VI0DVMRl3K2/h2CKQWVgH1j0+h8sfT8X6t7+vsv1x5j50C5jW4iz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH7SU1j/KcJNF609qTkJ8a+E8/Tb62U7KGhQpsvRdMwFXEpZ+vfIZhSUAnYt3966n59+fpg8q+vD1636s7J/nKdeW8pn1L9UWvP/HTVp8zP21mzuqfjc76/f9gg8rifas5z8P4KrawW6TxPv7VSsoeGCm2+FE3DVMSlnK1/h2BKQSVgt20a99MMPu4sUr3mnj9e3Fn96arTjZ93HL//lP3IrWMGAx/Pam7b8/RbKyV7aKjQ5kvRNExFXMrZ+ncIphRUAva9TaP6F97lv+58fHjTqP4dfP8pnRW2NY3q0ztn3Im87YnjmHO68tSep99aKdlDQ4U2X4qmYSriUs7Wv0MwpaASsHtvGrfaxWvn+PkV9niZ6a9TfTx4wOLT5249NI1tmU9APE+/tVKyh4YKbb4UTcNUxKWcrX+HYEpBJWDf3jRumz4mtPZx/7q8f/ygvbXr2DWNRfD9TWP8BsWeplHu9Dz91krJHhoqtPlSNA1TEZdytv4dgikFlYD9gqZxK653x+9ptC5nO3ctWgdMdo5cvi8u0to5R255mD+xulT/1B32eebj4P2nTBafOPE8/dZKyR4aKrT5UjQNUxGXcrb+HYIpBZWAfVfTOEGtS1vr56Kj5Hn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgD9A0OjcK7J6LjpLn6YddDSq0+VI0DVMRl3K2/h2CKQWVgN1700DR5Xn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgp2kgW3meftjVoEKbL0XTMBVxKWfr3yGYUlAJ2Fc3jddDn95/QgghdI0+fPymy80ghBBCDd0Lxru3bz5/+doX9zTQaj057tmwq0GFNl+KexqmIi7lbP07BFMKKgE7n55CtvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsNM0kK08Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH79qbR+bLq+eNV4usvMsnz9MOuBhXafCmahqmISzlb/w7BlIJKwL6raUzWOur7vDdr27f7Xf4a5Jbn6YddDSq0+VI0DVMRl3K2/h2CKQWVgP2wpvHQ2CY/XXxKuXK1urS+9nvtSatPqR5W/dcRYyPnGrQdV56nH3Y1qNDmS9E0TEVcytn6dwimFFQC9iM/PVW9/m49nl9w94+ZlJD5ChtOWt0z4m3QWKstzA9YzDCuPE8/7GpQoc2XommYiriUs/XvEEwpqATsZ3x66uHH2/yA/nNvP74Qb90cGDxp57ytnZ0W0Tc2X6d1wCBgRHmeftjVoEKbL0XTMBVxKWfr3yGYUlAJ2E9qGvMTb24aHZjBGwKdMjDubcTYSCCTRpGjXZTyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErB7bxojS+05cn8LGlynU1c6p04gz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH7rqbR+bDQ4geHyp3lv1afu7jU5KetI6tPrC41+MS+sQ0hLAKGk+fph10NKrT5UjQNUxGXcrb+HYIpBZWAfXvTQPvV6WZp5Hn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgp2lcqWS3L6ryPP2wq0GFNl+KpmEq4lLO1r9DMKWgErDTNJCtPE8/7GpQoc2XommYiriUs/XvEEwpqATsNA1kK8/TD7saVGjzpWgapiIu5Wz9OwRTCioB++qm8Xro0/tPCCGErtGHj990uRmEEEKooXvBePf2zecvX/vingZarSfHPRt2NajQ5ktxT8NUxKWcrX+HYEpBJWDn01PIVp6nH3Y1qNDmS9E0TEVcytn6dwimFFQCdpoGspXn6YddDSq0+VI0DVMRl3K2/h2CKQWVgJ2mgWzlefphV4MKbb4UTcNUxKWcrX+HYEpBJWDf2zSO+jqIC79QYtWpWwfv8Z/4yzRuvqcfdjWo0OZL0TRMRVzK2fp3CKYUVAL2XU3jwK+4nj99w4InX7LTLkbkefphV4MKbb4UTcNUxKWcrX+HYEpBJWDf3jQ618rVGx39nYv7F9ccX2r+xL7P6sGTZVvH91nGDdz3Xz4xa+V5+mFXgwptvhRNw1TEpZytf4dgSkElYD++acwvkVft7K/Tenprz9pT7z940MxaAx1q5/I8/bCrQYU2X4qmYSriUs7Wv0MwpaASsB/QNOZ/YX9f/cCm0boz0HpKa53+qY9qGq3bFMcaCCHP0w+7GlRo86VoGqYiLuVs/TsEUwoqAfth9zSsm0YLYPHv+y9vGnYGQsjz9MOuBhXafCmahqmISzlb/w7BlIJKwL69adx2l4pDmkZrwf46pk1jpHXQNBSUkj00VGjzpWgapiIu5Wz9OwRTCioB+66mcav958vbdrZKQnm1Pf8AUmvBxVP0i8Hag+fHzFdo7dxpwL88Tz/salChzZeiaZiKuJSz9e8QTCmoBOx7m4Z/hbs0TybP0w+7GlRo86VoGqYiLuVs/TsEUwoqATtNA9nK8/TDrgYV2nwpmoapiEs5W/8OwZSCSsCev2mga+V5+mFXgwptvhRNw1TEpZytf4dgSkElYKdpIFt5nn7Y1aBCmy9F0zAVcSln698hmFJQCdhXN43XQ5/ef0IIIXSNPnz8psvNIIQQQg3dC8a7t28+f/naF/c00Go9Oe7ZsKtBhTZfinsapiIu5Wz9OwRTCioBO5+eQrbyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErDTNJCtPE8/7GpQoc2XommYiriUs/XvEEwpqATsNA1kK8/TD7saVGjzpWgapiIu5Wz9OwRTCioB+/am0flu73HxZRfp5Xn6YVeDCm2+FE3DVMSlnK1/h2BKQSVg39U0XpfY0xYoKunlefphV4MKbb4UTcNUxKWcrX+HYEpBJWA/uGl0Hlfvgbz+62R/9fH8yPma81MccuMF7ZHn6YddDSq0+VI0DVMRl3K2/h2CKQWVgP2AT08NloRb7RbEpCf0Fxk8sn9GdLI8Tz/salChzZeiaZiKuJSz9e8QTCmoBOwH3NO4DZSE+U9Hjl+7s3qX4/KIxeV5+mFXgwptvhRNw1TEpZytf4dgSkElYD+jadxmn49qHXPIztb66BJ5nn7Y1aBCmy9F0zAVcSln698hmFJQCditmsbgjYVti9A0Asnz9MOuBhXafCmahqmISzlb/w7BlIJKwL6raVT/e+v7nupHmyanX7wHsupuCZ+ecijP0w+7GlRo86VoGqYiLuVs/TsEUwoqAfv2pjGuxTsPKLE8Tz/salChzZeiaZiKuJSz9e8QTCmoBOwnNY3WPQ2UXp6nH3Y1qNDmS9E0TEVcytn6dwimFFQC9jOaBlKW5+mHXQ0qtPlSNA1TEZdytv4dgikFlYCdpoFs5Xn6YVeDCm2+FE3DVMSlnK1/h2BKQSVgX900Xg99ev8JIYTQNfrw8ZsuN4MQQgg1dC8Y796++fzla1/c00Cr9eS4Z8OuBhXafCnuaZiKuJSz9e8QTCmoBOx8egrZyvP0w64GFdp8KZqGqYhLOVv/DsGUgkrATtNAtvI8/bCrQYU2X4qmYSriUs7Wv0MwpaASsNM0kK08Tz/salChzZeiaZiKuJSz9e8QTCmoBOy7mobpF2XsWbP/ZeToTHmeftjVoEKbL0XTMBVxKWfr3yGYUlAJ2Lc3jcm1u6tLeZqGH3meftjVoEKbL0XTMBVxKWfr3yGYUlAJ2I9vGpO7HJP7HtXbIOWP5gvO16zuaS3e+dHi6VpQaFyepx92NajQ5kvRNExFXMrZ+ncIphRUAnaTptHfWf3p/MjOfYnFU/TvaQyu0zePBuV5+mFXgwptvhRNw1TEpZytf4dgSkElYL+gaVTvGPSf0lnhnKZB2dgsz9MPuxpUaPOlaBqmIi7lbP07BFMKKgG7bdO4LX0manBn65jxA1bdBqlWC/rGNnmeftjVoEKbL0XTMBVxKWfr3yGYUlAJ2E+6p1Hds/nTUIc3jda9kc4iaFCepx92NajQ5kvRNExFXMrZ+ncIphRUAvbtTeM28J9Zl8fM96xqGouLtHb2f3Tf0zlda000Is/TD7saVGjzpWgapiIu5Wz9OwRTCioB+66msaj+fQafiujZszxPP+xqUKHNl6JpmIq4lLP17xBMKagE7OZNI9wNgYiePcvz9MOuBhXafCmahqmISzlb/w7BlIJKwG7bNBDyPP2wq0GFNl+KpmEq4lLO1r9DMKWgErDTNJCtPE8/7GpQoc2XommYiriUs/XvEEwpqATsq5vG66FP7z8hhBC6Rh8+ftPlZhBCCKGG7gXj3ds3n7987Yt7Gmi1nhz3bNjVoEKbL8U9DVMRl3K2/h2CKQWVgJ1PTyFbeZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYaRrIVp6nH3Y1qNDmS9E0TEVcytn6dwimFFQCdpoGspXn6YddDSq0+VI0DVMRl3K2/h2CKQWVgH1j03j48fa6lukXUPQX9/DdF6s8eDB8jjxPP+xqUKHNl6JpmIq4lLP17xBMKagE7LvuaUwul2kal3twKM/TD7saVGjzpWgapiIu5Wz9OwRTCioB+/FNY/Lt2vP7HvMDWo/Lg6uLtw5etfLIIi2o6ilaOzv3f6p+csjz9MOuBhXafCmahqmISzlb/w7BlIJKwH5w05hfSS9e+vd3Tq7Uq+fduXJVg+ddXLmsSYuGU8rz9MOuBhXafCmahqmISzlb/w7BlIJKwG716SmLplFdZ//Kk3U6Nx/6tzU6BWN+v2K+1OWjYCTP0w+7GlRo86VoGqYiLuVs/TsEUwoqAbvTpnGrfWCp+nj/yhsWGdw5cs9n0VV0eZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnYHTWNxQvx6uP9K2/wv7izdaLFOyrXToOFPE8/7GpQoc2XommYiriUs/XvEEwpqATspzaNW+M/fb7vqX48qbP4/OANK/fX6d8Y6XSY/hkH/SSQ5+mHXQ0qtPlSNA1TEZdytv4dgikFlYB9V9OwU79X+Fy5dRZxeZ5+2NWgQpsvRdMwFXEpZ+vfIZhSUAnY/TYNo7/pt1t5cparXlFv8jz9sKtBhTZfiqZhKuJSzta/QzCloBKwO20aKI08Tz/salChzZeiaZiKuJSz9e8QTCmoBOw0DWQrz9MPuxpUaPOlaBqmIi7lbP07BFMKKgH76qbxeujT+08IIYSu0YeP33S5GYQQQqihe8F49/bN5y9f++KeBlqtJ8c9G3Y1qNDmS3FPw1TEpZytf4dgSkGNsy/eLrhw49NTyFDi7/zLPQCVxnwpmoapiEs5W/8OwZSCGmc3rws7NpoGMpT4O/9yD0ClMV+KpmEq4lLO1r9DMKWgxtnN68KOjaaBDCX+zr/cA1BpzJeiaZiKuJSz9e8QTCmocXbzurBj29g0zvlm61Xrtw7eY5Jvxtgp8Xf+5R6ASmO+FE3DVMSlnK1/h2BKQY2zm9eFHdv2pvGK5/NanHbhROLv/Ms9AJXGfCmahqmISzlb/w7BlIIaZzevCzu2I5tG53H1Hkh1f7mndXz/4MmyreOrBjrPbRm477981BxK/J1/uQeg0pgvRdMwFXEpZ+vfIZhSUOPs5nVhx7b301OLTaNz96N//MjjVQd39uwx0AFE4u/8yz0AlcZ8KZqGqYhLOVv/DsGUghpnN68LO7btTeNO2L9G99Y0WrcpjjWAXiX+zr/cA1BpzJeiaZiKuJSz9e8QTCmocXbzurBjM28at/aHiy5vGnYG0KvE3/mXewAqjflSNA1TEZdytv4dgikFNc5uXhd2bCZNY/AzRec3jZHWQdM4UOLv/Ms9AJXGfCmahqmISzlb/w7BlIIaZzevCzu27U1j8qGjV933zO9pVD89Nf9RvxisPXjRScvD/LlrDaCb/Dv/cg9ApTFfiqZhKuJSzta/QzCloMbZzevCjm1j0xjX4C0FlFLi7/zLPQCVxnwpmoapiEs5W/8OwZSCGmc3rws7tjOaRueexuUvDzKV+Dv/cg9ApTFfiqZhKuJSzta/QzCloMbZzevCvm2kZvy+bBpsbIOb8sCkZA8NFdr8nk0WfNtGXHab/2z9OzxkS4mZEmpw88z+eqehfNA7+LVpsLGNb/fZEtxSsoeGCm1+zyYLvm0jLrvNf7b+HR6ypcRMCTW4uWXf0jT21Bo2NjY2NjY2NjY2NoWtrBbv3r559/bN5y9fO3r4yU9/dlktYou5vXv75moLl20p2UNDhTa/Z5MF37YRl93mP1v/Dg/ZUmKmhBrc3LL//v/fyrg7XGgmP/npz2y7D1u67fOXr1dbuGxLyR4aKrT5PZss+LaNuOw2/9n6d3jIlhIzJdTg5pmdpsFmu3mefustJXtoqNDm92yy4Ns24rLb/Gfr3+EhW0rMlFCDm3/2QYc0DbbVm//pt9tSsoeGCm1+zyYLvm0jLrvNf7b+HR6ypcRMCTW4+WenabBZbf6n325LyR4aKrT5PZss+LaNuOw2/9n6d3jIlhIzJdTg5p+dpsFmtfmffrstJXtoqNDm92yy4Ns24rLb/Gfr3+EhW0rMlFCDm392mgab1eZ/+u22lOyhoUKb37PJgm/biMtu85+tf4eHbCkxU0INbv7ZBx3+X4TKYvVvi0nfAAAAAElFTkSuQmCC" /><br />
<br />
3. And now the cool stuff begins. In the next phase we marked each box in which a trait and a skill intersected. For example, Task Planning requires a QFO to have a <span style="font-style: italic;">Fire and forget</span> attitude, posses a <span style="font-style: italic;">Broad Vision</span> of the team's goals and be <span style="font-style: italic;">Self Driven</span>. So we marked those intersection boxes with 1's. <span style="font-style: italic;">Self Improving</span>, on the other hand, is not especially important for Task Planning so we left that box empty. We proceeded to fill the table until it looked something like this:<br />
<img alt="" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABB4AAAFFCAIAAADEtbn3AAAgAElEQVR4nO2dQXLkSo5E8/6H0DF0DFnt6gJTu9nPMmchazVFBoMgAsh0OPxZWVsWFQz6A4MUIar6P/7nf//v4/PrKcQdHo/HuyO8DUr30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTffZjwodZCOMBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O5qLUQY+Ms9D0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTfXa2FCGO7mB4bjl99ZZJdpBcccSeecQjHAEee3S4WqVu7nJ2mOxljkhh3efEN3Xc4/O86ULzmnvBekpIcb/K7I25v/pPB80OEjxwm5Cbq29PLvoFajji8Xd/6nuJbM3cHLH7jtuwS/sixPpVaCxGG8Y7w4iRnW2KPMvwce9DXl9QhdWsNpBplh4dl+BiR9K2CgBJeiyEzdp933cMH9/VHqEDOEoY/pe2O+Pp9n66b4YtxPP07upHLr865vBBe1rD53MNj3J1NrYXwMF/ikx+czH/ecPzqz3cF4zeG4/bhnMMBZ7vsBlzeu+0B7FLHggx3PI4/C/D8XdWhlGWSSezhl4aVnE87iZ0XfnJcy9mcWJ9NbtG8nGdShLeAk+Rp6MQmi+Hy/F6OfxpWzmTLUGdRYRhvuO/ZX4dex13ORLYbz0p0tvFskknxLUZvZzHMUHN+3n8+7xbebphlUQ03Xs48Tzs/+8/RKnoeVoJvzQx32U442Ws+4NJxmHzuPp/qsjJnOXcTHhl+Va2F8HC5ds9GDj9P5h9Oe3mxzT/PN9p3eY7cLQHOTM9GDm9Yk2ktFdgdd3LEySRGtcu9jIKWNZYUfrJO7i4Yy5a7hzubc/iNYbhiz2p7Nn6Xc3i4yZbnO7CU7rhxfn0ZZ56Ud3K4y5DG3X0bzw59uRIsuwzHDFe7/RI4i3120Q1HThbwcT2fVWB4XUy2nE3uYHggy/q8nGeyi3GeyY7P6Wman3HLJeNbM/MDXdbkbMCt07HubpzfeC4m0/7aqNZCOHAv9/kN9PjVjBvNfOMw5K0blnGw8RDbLZe3MHfCyzLeDWw/9Jmg7yykhh+K+NbA/ITePdyl0XB3y5azMJMJ76Z9Ge4L83h9rV9xk8MNv3pWZ8vC9m0cBrZIDf86j7qefHis7RhLWkdNhkaTa8phdAvfUvQt6WGph6feOPNk2Qx3MUadfD470OWFcOs8XtZncvRb7mcVuzX//BQfGddKrYVwsH6pG+e8dQEbjz6/9/nuKZZ461KTG9+tG9PkbnI86OVXbw2w39TsR3xZ+OG0K2vg1veS4/19/v1juHE4yXzaW3bHww2/Oqx5Nm6v51XdAi/5y5WZeqDLa2E3+OyEzv96OcZdIsu1Nje63Dhf4ZZUxkl8RC3yW/McM7hndlwUl1HnM/guhFs57UvxVnL3VMZTYFyK4+OqtRAO1i9145wZNxr77vY5LfOvSN0a4AgwH7aYx5LNPeErw1/Obz8Fbh27hfEZ4rglcAHbL/8XEH5neG/Fog50durPYhzvGHf/ejnGXaLJeMte88vEl3B+aOMAO7duXFFLeh7j1syO2+nKDdZyRPuExgHu9WB39+W0TDg/xH83qrUQDuzfg4c36+FNbfjVyXV1Ns/ZEW8NONtoiXE5//wQk+LMS+GrwFFqKzvJNjmiby/3ETPCXx5r+8G9Bu5+vjyPlvkvt9jdfRvfxbCYlqVyLPvd0315To8b5zkv0xpX4OPknmY50G7A5K9z02GG4zyTQ88rMJnfcrjhFnvCy8pcHvcWl6vruTnFx6JdRprPPz/Fxpnnux83zqt9PPvDGSZzXk54meQ44LKMc6OznMMzPtk4nOFywh3DL6m1EB4m66wWk2vYsgsNpaXeG/41R7/1nWy78eyb6HHM2VTzAfbvYWcDXobjYhdHhqcPv57Gh7DhmMlelgvzcpKzjeuaHFBKGcF3H3/rUWshHOAvdyOOGzqN+5bSUu8N/5qjW56ExByVLg/8et5NWHS1FIpqh1LKCL67WgsRBv5yz4PSvbRU6fArtBX3oXLlgV9bR2vxxjdsbmqlNUIpZQTfXa2FCAN/uedB6V5aqnT4FdqK+1C58sCvLX7CECg1KaWM4LurtRBh4C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d7UWIgz85Z4HpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu+u1kKEgb/c86B0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3dVaiDDwl3selO6lpUqHX6GtuA+VKw/82uInDIFSk1LKCL67WgsRBv5yz4PSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBd1drIcLAX+55ULqXliodfoW24j5Urjzwa4ufMARKTUopI/juai1EGPjLPQ9K99JSpcOvcCmeXZlala+Vthb4tV1PiO/4LBLyLpRSRvDd1VqIMPCXex6U7qWlSodfIbu1GH/PqPlfKX5WS1sL/NqqtagLpZQRfHe1FiIM/OWeB6V7aanS4VfYPeX/MN+43X42lXHC7cj5/Aig5WECv7aWi+U5Xcz4js8KJ8IBpZQRfHe1FiIM/OWeB6V7aanS4Vc4E//ZPmwefBuHB90+b82nQgAqDBn4tbVcLIUW8xklQt6FUsoIvrtaCxEG/nLPg9K9tFTp8Ctc/nj1Ba3FcC/MH/RChSEDv7aBFwsyJULehVLKCL67WgsRBv5yz4PSvbRU6fArWJ7yozb6DgoFbDAC8GsbeLEgUyLkXSiljOC7q7UQYeAv9zwo3UtLlQ6/QkZrcfZzXN9BoYANRgB+bdVa1IVSygi+u1oLEQb+cs+D0r20VOnwK1h+x2P4i0mTjc/D49SxvGcjJ0kQgApDBn5tLRfLcDDmYj6jRMi7UEoZwXdXayHCwF/ueVC6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47motRBj4yz0PSvfSUqXDr9BW3IfKlQd+bfEThkCpSSllBN9drYUIA3+550HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzgu6u1EGHgL/c8KN1LS5UOv0JbcR8qVx74tcVPGAKlJqWUEXx3tRYiDPzlngele2mp0uFXaCvuQ+XKA7+2+AlDoNSklDKC767WQoSBv9zzoHQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDd1VqIMPCXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvrtaCxEG/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O5qLUQY+Ms9D0r30lKlw68QK05fRnrBN4Jf27OEr0+eekT8E+GAUsoIvrtaCxEG/nLPg9K9tFTp8Cu8sbWoWPOKmauAX9ttwrPPr09Sa/J3QSllBN9drYUIA3+550HpXlqqdPgVdk9IPxw3Dv96tuPkIexIhlcStdLWAr+2x6vge8vP/55dF2dbhgPOPluOGAL+iXBAKWUE312thQgDf7nnQeleWqp0+BUuH2t8Ay43Hv9agoqZq4Bf2/lz/257xtW060mOu4SAfyIcUEoZwXdXayHCwF/ueVC6l5YqHX4FtRa3qJi5Cvi1Nb5euLvRvtflPCHgnwgHlFJG8N3VWogw8Jd7HpTupaVKh19BrcUtKmauAn5t39VaPA+/+6TW4i6UUkbw3dVaiDDwl3selO6lpUqHXyG7tZj85kbFmlfMXAX82r6ytZj/ypNai7tQShnBd1drIcLAX+55ULqXliodfoXjT0YnPx/djhluHP6cdXiI3Zgq1EpbC/zaDpf9c9owuK+m+VRqLe5CKWUE312thQgDf7nnQeleWqp0+BVSn1H4UInywK/tuxK++CLFPxEOKKWM4LurtRBh4C/3PCjdS0uVDr+CWotbqER54Nf2ja3F8P1G3uFecJQXQyllBN9drYUIA3+550HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzgu49bi4/Pr4/Pr4cQQgghhBBCmPnz99/uj95aCA8P+E46D0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTffZhQrYXwgL/c86B0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3dVaiDDwl3selO6lpUqHX6GtuA+VKw/82uInDIFSk1LKCL67WgsRBv5yz4PSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBd79uLXb/MmN99sDxr5lKGOlcc0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTf3frWwmfyxtZCvJ4mp2/YY1d3Ly1VOnw4bcV9qFx54NcWP+Fd+twMKaWO4J9Qe8Lr1uLsPcZu4/HDfIbd57MB9o2TARML4aNDGX/W0nB7UUpLlQ6fQVtxHypXHvi1xU94i1Y3Q0qpHfgn9FbCe28t5v3D6TFGbcDZUbYDhn3LvJmZD8A5SdXpU0nkS91NaanS4WNpK+5D5coDv7b4CR00uRlSSg3BP6GRrcXxR/5nrwUuo5w95Q8PMd9rvnE4AO0k1aVPJfEvdQelpUqHj6WtuA+VKw/82uIndNDkZkgpNQT/hIa1FmcP7s/RKwLfW4vFLuJyql1UsU6fYuJf6g5KS5UOH0tbcR8qVx74tcVP6KDJzZBSagj+CX1Fa/E8PN9jthZo54aAPiXFv9QdlJYqHT6WtuI+VK488GuLn9BBk5shpdQQ/BMa1lo8z38hyvgcf9z9OGx4iPnkjrcWaCepLn0qiX+pOygtVTp8LG3FfahceeDXFj+hgyY3Q0qpIfgn1N9avCDKi5m/eBEOOpTx8Zvt9jemWqS0VOnwGbQV96Fy5YFfW/yEt2h1M6SU2oF/Qm8l7NJa6K1FLJ0rSeleWqp0+BXaivtQufLAry1+whAoNSmljOC7v621EHzgL/c8KN1LS5UOv0JbcR8qVx74tcVPGAKlJqWUEXx3tRYiDPzlngele2mp0uFXaCvuQ+XKA7+2+AlDoNSklDKC7z5uLT4+vz4+vx5CCCGEEEIIYebP33+7P3prITw84DvpPCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF892FCtRbCA/5yz4PSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBd1drIcLAX+55ULqXliodfoW24j5Urjzwa4ufMARKTUopI/juai1EGPjLPQ9K99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE3/26tdj9y4z12d3DXjyVuEvn4lO6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47ta3Fj6T17QW+FV+AQhFQMjwLqq7D39wUFqqdPgV2or7ULnywK8tfsK78N3Jz6CUOlL0hDpbi7P3GLuNxw/zSbbDtht/xgx32Q2Y5PkZcPYGxuhl3GIZtstjnGE38iz2i3nv0d9LaffhtTPcUojS4VdoK+5D5coDv7b4CW9BeSc/g1JqR90TOn7AvvXWYveUfPzS2UP8fIbJtJa9JrNdznwZYL7l7o7H8ZcFmY98FwgZ3gWBe8X714TS4VdoK+5D5coDv7b4CR2Q3cnPoJQaUvGE+luL40/Kz37ifnngW63F8LjzaS8Pse71+M2tHSchz6ade70LhAzvgsC94v1rQunwK7QV96Fy5YFfW/yEDsju5GdQSg2peEKdrcXkAf34EHz2BH/8fKsZyGgtjF7HLZMzbd9x2FoMJ5zs8kYQMrwLAveK968JpcOv0Fbch8qVB35t8RM6ILuTn0EpNaTiCY1vLZ6H513H8/Hlw/dwr5XBd786mfaIcUe1FnUhcK94/5pQOvwKbcV9qFx54NcWP6EDsjv5GZRSQyqeUGdr8Tz/xSH7g+9xhp+Nk32Ne11Gsqc6O9xlMMuOl+UaTnu2+3sX3HuP/l4I3I8KpaVKh1+hrbgPlSsP/NriJ3RAdic/g1JqSMUTOn661n8yTzjAX+55lHZ//Ga7/Y2pFikdfoW24j5Urjzwa4uf8BaUd/IzKKV21D2hai1EGPjLPQ9K99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE312thQgDf7nnQeleWqp0+BXaivtQufLAry1+whAoNSmljOC7q7UQYeAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfPdxa/Hx+fXx+fUQQgghhBBCCDN//v7b/dFbC+HhAd9J50HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzguw8TqrUQHvCXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvrtaCxEG/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7W1uL7n2XEHymzQBmBFyfEXxArcNvNoXQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDdTa3Fz6AVn9fUYnuU7CPemh9/KazTwfGM6u7DvreKVOnw4bQV96Fy5YFfW/yEd+lzM6SUMoLvfq+1OP71+/PP/x5H/mwcjhlOdTbJ8ILZbt+NtKQ623KZ7ZhqMu0x2NkR5xOebQQBM9VrKO2+vV6O28EpHT6DtuI+VK488GuLn/AWrW6GlFJG8N2HCce/EDX5vHsgHk59OWY+YH61OCacT+6eyjJgeJRjuYw5cQCPlwqBe+lvSKXDx9JW3IfKlQd+bfETOmhyM6SUMoLvbm0tnqOH3VvP67cevqNai+GALWeT38pmmXbYaE0ynE2IvKSQs2VD4F76G1Lp8LG0FfehcuWBX1v8hA6a3AwppYzgu99oLZ6Hp+oXtBbP8+fpxV7lbCpLnvkA+wyTYJOlA9tgYKZ6DQTupb8hlQ4fS1txHypXHvi1xU/ooMnNkFLKCL67qbW42wkMpw7sBMInrNVaXH71XWCmeg0E7qW/IZUOH0tbcR8qVx74tcVP6KDJzZBSygi+u/WtxfEn5fYXDrtXHE/D0/nZJMf0w2CTI273svcb86ks0w5D7sIfdz/bckyLAGywF0DgXvobUunwsbQV96Fy5YFfW/yEDprcDCmljOC7W1sLy24ZTJ7OBRqdT1Bp98dvttvfmMpI6fAZtBX3oXLlgV9b/IS3aHUzpJQygu9eoLUA/1G9+KHzOaJ0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3Z2thRBH8Jd7HpTupaVKh1+hrbgPlSsP/NriJwyBUpNSygi+u1oLEQb+cs+D0r20VOnwK7QV96Fy5YFfW/yEIVBqUkoZwXcftxYfn18fn18PIYQQQgghhDDz5++/3R+9tRAeHvCddB6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvvswoVoL4QF/uedB6V5aqnT4FdqK+1C58sCvLX7CECg1KaWM4LurtRBh4C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d7UWIgz85Z4HpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu9uai22/zLjLYFECTqfO0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTf3dpazHfICCHK0eQ8Dnvs6u6lpUqHD6etuA+VKw/82uInvEufmyGllBF896XW4vgq4+z9xnzk9qvHY01mw69vKzqcjuMq3W4vSmmp0uEzaCvuQ+XKA7+2+Alv0epmSCllBN/d31pkbDx+Po4fziAQ6HNGKO/dpaVKh4+lrbgPlSsP/NriJ3TQ5GZIKWUE393aWly+djhOF9JaTHbEL243+pwRynt3aanS4WNpK+5D5coDv7b4CR00uRlSShnBd7/31uLsuX+4Mam1eJ78NqF4O31OCuW9u7RU6fCxtBX3oXLlgV9b/IQOmtwMKaWM4LvH/EKUfWTgWwuBRp9TQ3nvLi1VOnwsbcV9qFx54NcWP6GDJjdDSikj+O7+1uI5+kWp4a9ObbefHfjnq/a3FvjF7UafM0J57y4tVTp8LG3FfahceeDXFj+hgyY3Q0opI/juptZicbpYLn8pS7yLDqfj8Zvt9jemWqS0VOnwGbQV96Fy5YFfW/yEt2h1M6SUMoLvXrK10FsLTDqfEUr30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTfPbi1EJ3BX+55ULqXliodfoW24j5Urjzwa4ufMARKTUopI/juai1EGPjLPQ9K99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE333cWnx8fn18fj2EEEIIIYQQwsyfv/92f/TWQnh4wHfSeVC6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47sOEai2EB/zlngele2mp0uFXaCvuQ+XKA7+2+AlDoNSklDKC767WQoSBv9zzoHQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDd1VqIMPCXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvruptdj+ywzjXCvm+FV7/qcm706BReeCULqXliodfoW24j5Urjzwa4ufMARKTUopI/ju1tbi7lz45iv47Lhr8mwg+M2wq6zuXlqqdPhw2opfonXyYvBri5/wLn0WOaWUEXx3f2txfI8xaS12g+d7DbecTXjcfvx8+dblzOXsKj3qHHc/G3PmOKzSRHwo8l4wU8VyPH3b7UUpLVU6fAZtxedonbwe/NriJ7xFq0VOKWUE332Y8PoXos6ejIfzDh/Z7V89zj88oiPYfJe7RxmOtMTwDQAEPF4glPfu0lKlw8fSVtyC1skrwa8tfkIHTRY5pZQRfHdrazHZzdFa/Hw4/q993/VgUbscW6/d9rteZ9PuDoEGcrZYKO/dpaVKh4+lrbgFrZNXgl9b/IQOmixySikj+O4lW4vn+UP2W1qLxRjGg26/hLmwMFNlQHnvLi1VOnwsbcUtaJ28Evza4id00GSRU0oZwXd/W2ux8nn+2P3zoURrYc95Nj8OmKkyoLx3l5YqHT6WtuIWtE5eCX5t8RM6aLLIKaWM4Ls7W4vn5jd2hsN2uxwHT8bb31pMgm3HPzYcx991mdgNEx43TnLOp51bvB3YYOFQ3rtLS5UOH0tbcQtaJ68Ev7b4CR00WeSUUkbw3U2tBRrz5/75eJFHhzo/frPd/sZUi5SWKh0+g7bic7ROXg9+bfET3qLVIqeUMoLvXrW1uPXze/zTwEHnOlO6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47iVbC4EJ/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7j1uLj8+vj8+shhBBCCCGEEGb+/P23+6O3FsLDA76TzoPSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBdx8mVGshPOAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfHe1FiIM/OWeB6V7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvrtZChIG/3POgdC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N2trcXPv8x4VTA/xpBj8wqCsHSuHqV7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvbmottoPuKmWPd8+Gf27K0bmklO6lpUqHX6GtuA+VKw/82uInvMvwR8DVNSmljBR1v91abDfunL8/bzdu/4+ohnvtdj+OHx7urNXZfXV+9GPas9l2YSYhm9O5IJTupaVKh1+hrbgPlSsP/NriJ7zF8QFpu70olFJG6ro7W4vh4/iwZ5g/uD8PtTtrYy6Pfpxq8tmYdjIA/9S+ns41oXQvLVU6/AptxX2oXHng1xY/oYOKT6KXUEoZqege3Fpcbhz+yH/SAGx3PAY4/u9kNkt/YmyEznI2p3NNKN1LS5UOv0JbcR8qVx74tcVP6KDik+gllFJGKrq/urW4DDEp2fE1wutbi10MsaVzWSjdS0uVDr9CW3EfKlce+LXFT+ig4pPoJZRSRiq6324tds/xz6vncssbibPx87grn++m1csKC52LQ+leWqp0+BXaivtQufLAry1+QgcVn0QvoZQyUtHd1Fo8N7/LNN9y9nn7aL7d8ezx/dgGDDdefj47+uTQRh38U/t6OteE0r20VOnwK7QV96Fy5YFfW/yEDio+iV5CKWWkoru1tRDfXL6E6UznglC6l5YqHX6FtuI+VK488GuLn/AWj99st78x1SKUUkbququ1uMfwNItvOteE0r20VOnwK7QV96Fy5YFfW/yEIVBqUkoZwXdXayHCwF/ueVC6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47motRBj4yz0PSvfSUqXDr9BW3IfKlQd+bfEThkCpSSllBN993Fp8fH59fH49hBBCCCGEEMLMn7//dn/01kJ4eMB30nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7DhGothAf85Z4HpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu+u1kKEgb/c86B0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3dVaiDDwl3selO6lpUqHX6GtuA+VKw/82uInDIFSk1LKCL77dWux+5cZ67Nn7Hs2+GUBxLN3xSjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d+tbC5/JW/zVTryLztWr7j78wUFpqdLhV2grfgnfIgcHv7b4CUOortn5yi3q7mwtzt5j7DYeP8xn2G6Zz3w2eDft2fhhgMm+ZwGeJye+J53rUNp9uOCHWwpROvwKbcXnUC5ycPBri58whNKana/cuu7O1mL4pTP/8TEOz+hnG0MGz0XcASbuDelcBAL3ivevCaXDr9BW3ALZIgcHv7b4CUMg0Ox85VZ097cWZz/1P465PPDLWouzFxGxAdrSuQgE7hXvXxNKh1+hrbgFskUODn5t8ROGQKDZ+cqt6O5sLeaP77uHb5y3FrFzqrXY0bkIBO4V718TSodfoa24BbJFDg5+bfEThkCg2fnKrege31o8Dw/fIK2FRUGtxQvQ42oAAB0ESURBVAqdi0DgXvH+NaF0+BXailsgW+Tg4NcWP2EIBJqdr9yK7s7W4nn+C1GXT+Fnuz+nD+uOwccxxxnONi4G6EnnIhC4V7x/TSgdfoW24hbIFjk4+LXFTxgCgWbnK7eiu7W1eM2BRWk6n9PS7o/fbLe/MdUipcOv0FZ8DuUiBwe/tvgJQyit2fnKreuu1kKE0fmcUrqXliodfoW24j5Urjzwa4ufMARKTUopI/jub2stBB/4yz0PSvfSUqXDr9BW3IfKlQd+bfEThkCpSSllBN9drYUIA3+550HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzgu49bi4/Pr4/Pr4cQQgghhBBCmPnz99/uj95aCA8P+E46D0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTffZhQrYXwgL/c86B0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3dVaiDDwl3selO6lpUqHX6GtuA+VKw/82uInDIFSk1LKCL67WgsRBv5yz4PSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBdze1Ftt/mbE072vLMTkc/ompSOeqUrqXliodfoW24j5Urjzwa4ufMARKTUopI/ju1tZivoN1XpjWQmTQueDV3Yc/OKgiVTr8Cm3FA1G58oCqbZOLpYnmk1TKCJS7fcldtxa7lxjHdxrDFx27g50N+Nk4eU8yOeLxcMNDnB10riAmdC5UaffjRbHdDk7p8Cu0FY9F5coDp7ZNLpYmmt9QShnBcb+15EytxeSrtzbuBhwHW0LPD3eZf7ILzinEp3OtCNxLf0MqHX6FtuJRqFx5oNW2ycXSVrMPaO7+1mL38/uQ1sI47TD08Q2DJYMjP9opRKZzrQjcS39DKh1+hbbiUahceaDVtsnF0lazD2juAW8tjlvW31rYp9oNvns4e2uxm19Y6FwuAvfS35BKh1+hrXgUKlceaLVtcrG01ewDmjtVa3HrcJbMxoOKMzoXjcC99Dek0uFXaCsehcqVB1ptm1wsbTX7gOae0lo8N7/ddNwyaSeOY4xP+ceZzzKcZba/tUA7hch0rhWBe+lvSKXDr9BWPAqVKw+02ja5WNpq9gHN3dlaNOTyBYs40rlQpd0fv9luf2MqI6XDr9BWPBaVKw+c2ja5WJpofkMpZQTH/daSU2sxfusi5nSuFaV7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvrtZChIG/3POgdC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N3VWogw8Jd7HpTupaVKh1+hrbgPlSsP/NriJwyBUpNSygi++7i1+Pj8+vj8egghhBBCCCGEmT9//+3+6K2F8PCA76TzoHQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDdhwnVWggP+Ms9D0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTfXa2FCAN/uedB6V5aqnT4FdqK+1C58sCvLX7CECg1KaWM4LurtRBh4C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d2tr8f3PMsKP9zPtyuSBVcY/YSvcsnOUgrt6cyjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d1Nr8TMo1geknXjBtA4ykmTb4VTv9VC6l5YqHX6FtuKXDH9ApnKFgF9b/IQhNNF8kkodwT+h9oSnrcVuruOMlxu3Xcpw+3CX3bT2fbf5h/4THbvLzmt7LEuJjiMvy3tpNxk5Sb7LMI99ZP5VbijdS0uVDr9CW/E5w28Wwy3iLvi1xU8YQhPNbyilduCf0FsJx78QdfbXywfWyx0nn8/i3ppn8sQc7jI8luWZfn6sI0Y7YwxjWYyRGkLpXlqqdPgV2opbQP4mXR382uInDKGtJiv4J9TfWjzPnzWPP/ae//D71ufjzI55Lic37rLeX12W6LK1uFWKy5HD/uRsqmGes126QeleWqp0+BXailvA/yZdF/za4icMoa0mK/gndKm1eJ48+M6fhtdbi1vdAn5r8TzgK6Y9qqW1+Nk4CXDZYKAt91dC6V5aqnT4FdqKW8D/Jl0X/NriJwyhrSYr+CfU2VrMH5HVWszfDExmsB9rJaqlNXIEuPtVbijdS0uVDr9CW3EL+N+k64JfW/yEIbTVZAX/hDpbi+fhJ9aPDcdhZ4OfJ4+8xxy7SSbTXu47P5Ax9jzYdlrLg/5xkrOowxN2a/ezkWeDjV8dcjmAGEr30lKlw6/QVtzC8I76liR84NcWP2EIbTVZwT+hxoTX/8k8NDFxl4wz2HlVULqXliodfoW24nMev9luf2MqDvBri58whCaa31BK7cA/obcSqrXgR61FLJTupaVKh1+hrbgPlSsP/NriJwyBUpNSygi+u7O1EOII/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7j1uLj8+vj8+shhBBCCCGEEGb+/P23+6O3FsLDA76TzoPSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBdx8mVGshPOAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfHe1FiIM/OWeB6V7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvrtZChIG/3POgdC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N2vW4vdiPlfz2acbH87K8FeIwVbuh1VcmZA6V5aqnT4FdqK+1C58sCvLX7CECg1KaWM4LvHtxb2rxJg76xuTbI+ZxLNT/cESvfSUqXDr9BW3IfKFcL3/yfMceNbwtjBT3iXoifCAaWUEXx30y9E/Qz6/rD76/M/q/n41eH27TzHRmW7y/BLZ1uOhxvucjzuZM7hlknO4/b5MKOyPc9RZyg+n+GsVpd2R4U+ULqXliodfoW24j5UrnWO33q225HBT3iLuifCAaWUEXz3YUJPazGccfJ5Msn8wpgfdOzz+1HYfjj7cedRjYe7nMcy4dkYY8HttWpy/zJC6V5aqnT4FdqK+1C5oqj4HQE/oYOKJ8IBpZQRfPcbrcXwR9rHWdyfw1uL4U/Z57tMZvC1FsPdJ0dcVPbtaNc82l2KNIHSvbRU6fArtBX3oXJFUfGJFj+hg4onwgGllBF8d6DW4jl6Wp2MPz7X2ruX+TzDz8YBl7sf0yK0Fo4ARpEmULqXliodfoW24j5UrigqPtHiJ3RQ8UQ4oJQygu9uai2G4wLbiUkU43hjvLvz5LUWZwEW8++2v6C1uPxqEyjdS0uVDr9CW3EfKlcUFZ9o8RM6qHgiHFBKGcF3z20tnpsHXPtbi7Pn192Xhgcaznartbic5GzjUfksw3HH4VTzQ0/cjzW3i8932U0+PyOtoHQvLVU6/AptxX2oXFEMv4W9JYkd/IQOKp4IB5RSRvDdx8/Gb/lP5q08pOoBF4HOlad0Ly1VOvwKbcV9qFzrPH6z3f7GVBbwE96i7olwQCllBN8dq7UY/kg+e18RRefiU7qXliodfoW24j5Urjzwa4ufMARKTUopI/juQK2FqA7+cs+D0r20VOnwK7QV96Fy5YFfW/yEIVBqUkoZwXdXayHCwF/ueVC6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47uPW4uPz6+Pz6yGEEEIIIYQQZv78/bf7o7cWwsMDvpPOg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3HyZUayE84C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d7UWIgz85Z4HpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu+u1kKEgb/c86B0Ly1VOvwKbcV9qFx54NcWP2EIlJqUUkbw3U2txfZfZuw2Hj+vH14UpfPZpHQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDdra3FZLe7kiFFcUyCfzKq07nClO6lpUqHX6Gt+CXDH4GpXCHg1xY/YQhNNJ+kUkag3O1L7qK1eJyw++rlLtuZh73KcKrdbJaDDncZDhv+1RLMcixj7LrQiDigdC8tVTr8Cm3F5+y+4+y2ixXwa4ufMIQmmt9QShnBcb+15K5/IWr4wH32+fiEPR9zzDrPbTnocIslmzHY2Zk+DrisYV04LHxQupeWKh1+hbbiFpo8db0F/NriJwyhrWYf0Nz9rcVkt+EP78/6kPm+z99P3sepbh10ctyzjZO2YR7sOM/ZAKNgRTgsfFC6l5YqHX6FtuIWmjx1vQX82uInDKGtZh/Q3F/RWlwe1d5ajCVsB/350nxCX2thz3NsgSyxi8JnZIfSvbRU6fArtBW30OSp6y3g1xY/YQhtNfuA5g7dWlimWhm53vYY55n0J5NDE8BnZIfSvbRU6fArtBW30OE+/C7wa4ufMIS2mn1Ac/e3Frvf25m3B8df8jnuvnsVYJ9q99VJwuOOw6mMO86DOYpwKVgODgsflO6lpUqHX6GtuIXh/f8tSfjAry1+whDaavYBzd245PSfzItn11G8MUkerF4WKN1LS5UOv0Jb8TmP32y3vzEVB/i1xU8YQhPNbyiljOC431pyai3iIXtBMYRY7RJK99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE312thQgDf7nnQeleWqp0+BXaivtQufLAry1+whAoNSmljOC7q7UQYeAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfPdxa/Hx+fXx+fUQQgghhBBCCDN//v7b/dFbC+HhAd9J50HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzguw8TqrUQHvCXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvrtaCxEG/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O43Wouff5yRccjXcOvQZ4NX8uMviBW47eZQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7W1mI7btHquLtjwhdXVu2EhT6mR6q7D39qUFqqdPgV2opfwrfIccCvLX7CPKprNj93Fd1NrcVEY/gqY77xcvvlnPapjjvOcw4H76Y9Gz93sQd4niwmfCpmjqK0+3BxDrcUonT4FdqKz6Fc5CDg1xY/YSqlNTufu7ruw4TW1uL4THxr43yes91nue8cen2wMczdABNrcMoFDoTAveL9a0Lp8Cu0FbdAtsihwK8tfsIkCDTbnrtnTfd7rcXxR/LDMcaNwy2P32w3Xud+R2tx/KulSncDlKBc4EAI3CvevyaUDr9CW3ELZIscCvza4idMgkCz7bl71nS/11rs/prUWkyyzn+i//bWIi9ACcoFDoTAveL9a0Lp8Cu0FbdAtsihwK8tfsIkCDTbnrtnTXdTa/G88xyc11qcTegIGTL4GEOtxbsjvA0C94r3rwmlw6/QVtwC2SKHAr+2+AmTINBse+6eNd2trcVz86s+ixvPuoLt4/Vu5GTCy0PMO4G7g49jjjOcbVwMgE+5wIEQuFe8f00oHX6FtuIWyBY5FPi1xU+YBIFm23P3rOk+fs4v+p/Mwy83N53rX9r98Zvt9jemWqR0+BXais+hXOQg4NcWP2EqpTU7n7u67motRBid60/pXlqqdPgV2or7ULnywK8tfsIQKDUppYzgu1O1FuK94C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d7UWIgz85Z4HpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu8+bi0+Pr8+Pr8eQgghhBBCCGHmz99/uz96ayE8POA76Two3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfPdhQrUWwgP+cs+D0r20VOnwK7QV96Fy5YFfW/yEIVBqUkoZwXdXayHCwF/ueVC6l5YqHX6FtuI+VK488GuLnzAESk1KKSP47motRBj4yz0PSvfSUqXDr9BW3IfKlQd+bfEThkCpSSllBN/d1Fr8/LOM8CMJJjqfYkr30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTf3dpaTEavHOllu4sX0PkcUbqXliodfoW24j5UrhCGP3yEqi1+whAoNSmlVoByt58dU2sx+Tx8y/H919324efjyOOcx0OEvFoRK3QuPqV7aanS4VdoK+5D5Vrn51v2cDsC+AlDoNSklFoEx/3W2Tn9hShjV3B2pPmwXdtgGTk/ongxnU8BpXtpqdLhV2gr7kPligL/4Q8/YQiUmpRSbtDcV99aPA1dgeVIxkkmG4fvMY4m4pV0PgWU7qWlSodfoa24D5UrCvyHP/yEIVBqUkq5QXNPby2eJ793dWuSW4ebbxSvpPMpoHQvLVU6/AptxX2oXFHgP/zhJwyBUpNSyg2ae0prYXx14JtErUUhOp8CSvfSUqXDr9BW3IfKFQX+wx9+whAoNSml3KC5+1uL3W8fbbc/D8/9wxcXl285br0POUZCq3VDOp8CSvfSUqXDr9BW3IfKFQX+wx9+whAoNSml3KC5O1sL3wHQ5EU2nc84pXtpqdLhV2gr7kPlWufxm+32N6bagp8wBEpNSqlFcNxvnZ2l1uJ4DNGEzied0r20VOnwK7QV96Fy5YFfW/yEIVBqUkoZwXcPbi1EZ/CXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvrtaCxEG/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3H7cWH59fH59fDyGEEEIIIYQw8+fvv90fvbUQHh7wnXQelO6lpUqHX6GtuA+VKw/82uInDIFSk1LKCL77MKFaC+EBf7nnQeleWqp0+BXaivtQufLAry1+whAoNSmljOC7q7UQYeAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfHe1FiIM/OWeB6V7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvbm0ttv844zUh3PviF52VzpWndC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N1NrcVuEJSVWgscOlee0r20VOnwK7QV96FyhTD8sSN+bfET3qXoiXBAKWUE332ptdi9x9i92Ri+6Nh+aTj/cJfj1TJ5izI57uRwZ1LCTueiUbqXliodfoW24j5UrnV+vq0PtyODn/AWdU+EA0opI/juq63FfOPwq8eRkyvh8hDzq8g4zzy8MNK5epTupaVKh1+hrbgPlSuKik+0+AkdVDwRDiiljOC757YWw3cC810mM7ymtcA/Z7B0Lh2le2mp0uFXaCvuQ+WKouITLX5CBxVPhANKKSP47mGtxfPq15yMG8/G2AfcetEx7CXUYPjoXDRK99JSpcOv0Fbch8oVRcUnWvyEDiqeCAeUUkbw3ePfWgy33HrRcZ1v4bhnbz8mkwgjnYtG6V5aqnT4FdqK+1C5oqj4RIuf0EHFE+GAUsoIvruptXiOfrXpuPNwwG6jpbW4nORs4/xLP1smhzubU1joXDdK99JSpcOv0Fbch8oVRcUnWvyEDiqeCAeUUkbw3a2txa2J8LW/qZgZmc41pHQvLVU6/AptxX2oXOs8frPd/sZUFvAT3qLuiXBAKWUE3z2ytSj3I/+KmZHpXEZK99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE3z2stRACf7nnQeleWqp0+BXaivtQufLAry1+whAoNSmljOC7q7UQYeAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfPdxa/Hx+fXx+fUQQgghhBBCCDN//v7b/dFbC+HhAd9J50HpXlqqdPgV2or7ULnywK8tfsIQKDUppYzguw8TqrUQHvCXex6U7qWlSodfoa24D5UrD/za4icMgVKTUsoIvrtaCxEG/nLPg9K9tFTp8Cu0FfehcuWBX1v8hCFQalJKGcF3V2shwsBf7nlQupeWKh1+hbbiPlSuPPBri58wBEpNSikj+O7XrcXuX2ZMdkvNZPzqa7iVASHwa+hjeoTSvbRU6fArtBX3oXLlgV9b/IQhUGpSShnBd7e+tdiNU2vx7giIdC5LdfftDw62G98S5i6lw4fTVtyHyhUC/jWInzCEJppPUqkj+CfUntDaWuxmPL7ZOA44+7wdPJz8bPCtmS2TnEkND3G2cVeH4aGPatWhlDJS2n173R23g1M6fAZtxX2oXOvgX4P4CUNoovkNpdQO/BN6K6GptTg+Ol8+68837h7Nz/KtzDzEeNzLmS0lxlkQGXDbzSFwR75/XVI6fCxtxX2oXFHgX4P4CUNoq8kK/gmNbC2OnwNbC2Pildbi+Dpikv9y5LA/OZvqGIYDYrVLCNzx718TSoePpa24D5UrCvxrED9hCG01WcE/oYitxfPwwB3VWhxndkxi3Hisz6SxYW0wKKWMELjj378mlA4fS1txHypXFPjXIH7CENpqsoJ/QlFai8snb2PiuzM78l9uPDvQPAbaygiBUsoIgTv+/WtC6fCxtBX3oXJFgX8N4icMoa0mK/gn9HWtxXPze0THjccZzlqCsx/5+2aezzNpjSYjL49ozEMAq5cFAnf8+9eE0uFjaSvuQ+WKAv8axE8YQltNVvBPqL+1yGPeSGDOfHaU5nQuRWn3x2+229+Yykjp8Bm0Ffehcq2Dfw3iJwyhieY3lFI78E/orYSvbi2OscBn3h0lb/JadC4FpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu/+/tZC0IC/3POgdC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N3VWogw8Jd7HpTupaVKh1+hrbgPlSsP/NriJwyBUpNSygi++7i1+Pj8+vj8egghhBBCCCGEmT9//+3+6K2F8PCA76TzoHQvLVU6/AptxX2oXHng1xY/YQiUmpRSRvDdhwnVWggP+Ms9D0r30lKlw6/QVtyHypUHfm3xE4ZAqUkpZQTfXa2FCAN/uedB6V5aqnT4FdqK+1C58sCvLX7CECg1KaWM4LurtRBh4C/3PCjdS0uVDr9CW3EfKlce+LXFTxgCpSallBF8d1Nr8fPPMl4f5e7glZD4ZwuczgWkdC8tVTr8Cm3FfahceeDXFj9hCJSalFJG8N2trcVk9NtROwFC52JWdx/+7KCKVOnw4bQVv0Tr5MXg1xY/YQjVNXXl7sB397cWk8/DtxzD7dstZ+Png3fTno0fBpjsexbgebLKxbPCcs+jtPtw/Q+3AFI6fAZtxedonbwe/NriJwyhtKau3CP47tbW4uw5e/f5+ME43vL51uALQ2+AiaDoXBMC99L37tLhY2krbkHr5JXg1xY/YQgEmrpyt+C7W1uL4+cSrcXZi4jYAOKbzjUhcC997y4dPpa24ha0Tl4Jfm3xE4ZAoKkrdwu+e3Br8Tz/faG3txZ5AcQ3nWtC4F763l06fCxtxS1onbwS/NriJwyBQFNX7hZ894DWwvhrQq9vLSxthlqLQDrXhMC99L27dPhY2opb0Dp5Jfi1xU8YAoGmrtwt+O7W1mL3e0Tb7c/DM/fwxcVwknkncHfwZZKzDMd97wYQz941IXAvfe8uHT6WtuIWtE5eCX5t8ROGQKCpK3cLvruptfBNN3lpICjpfIpLuz9+s93+xlRGSofPoK34HK2T14NfW/yEIZTW1JV7BN89vrU4roDJkQQTnU8xpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu8e3FqIzuAv9zwo3UtLlQ6/QltxHypXHvi1xU8YAqUmpZQRfHe1FiIM/OWeB6V7aanS4VdoK+5D5coDv7b4CUOg1KSUMoLvPm4tPj6/Pj6/HkIIIYQQQghh5s/ff7s/emshPPz5++/dEd4GpXtpqdLhV2gr7kPlygO/tvgJQ6DUpJQygu8+TKjWQnjAX+55ULqXliodfoW24j5Urjzwa4ufMARKTUopI/juai1EGPjLPQ9K99JSpcOv0Fbch8qVB35t8ROGQKlJKWUE332Y8P8Bmnc6ApnqfCYAAAAASUVORK5CYII=" /><br />
<br />
4. In the next phase, we summed (in the last row) the number time each trait was used:<br />
<br />
<img alt="" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABBwAAAFaCAIAAAAywNmEAAAgAElEQVR4nO2dYZLcPK5sa/+LeMu4y3jhf16B11H3h2f6yiJFQSBQlUjkCcdEWS1ReUBKLbTa873+3//8/7cQT3i9Xt+O8DUo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfPcx4UtNhXgK/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3V1MhAsBf6HlQupeWKh1+h7biPlSuPPBri58wBEpNSikj+O5qKkQAx2X0OjB+9ZNJTpE+cMaTeMYpHDs48pwOsUg9OuRqmp5kjEliPOTDt3Lf6fC/30DxmXvCd0lKMt7kT2c83vwXO69PEb7nNCE3Ud+ePvYN1HLG6e360fcU35p5usPmN27LIeGPHPtDqakQARjvBR9OcrUl9izTz7En/XxJHVKP1kCqUXZ4WKYPEBnfJDgo4bUZMuPwdb89fWTff3gK5Cph+PPZ6YyfP/btuhl+GMdzv6MPuf3qmtsL4WOtms89PMaj0dRUiMesF/fihyXrnzGMX/35fmD8ljBun4453eHqkNMOt3dtewC71FiQ6YHj/lcB3v9WdSplGWQRe/qlaSXXwy5i54VfnNcymwvrq8EtmrfjLIrwFXCSvA092GIx3M7v7f5vw8pZbJnqbCpM402Pvfrr1Gs85ErkuPGqRFcbrwZZFN9i9HU2w0w11/P+8/m08E67WRbVdOPtyOu069l/z1bRe1gJvjUzPeQ44OKo9Q63jtPka/f1ULeVucp5GnBkkkpNhXjK7aq92nP6eTH+dNjby2z9eb3Rfsh75m4JcGV6tef0VrUY1lKB03kXZ1wMYlS7PcooaFljSeEX6+TpgrFseXq6qzGn3xKmK/aqtlf7n3JOT7fY8v4GltKNG9fXl3HkRXkXp7sNaTzct/Hq1LcrwXLIdJ/pardfAlexry666Z6LBTyu56sKTK+LxZarwR1MT2RZn7fjLA4xjrM48L2cpvWMWy4Z35pZn+i2Jlc7PJqOfXfj+Ma5WAz7f1vUVIinuBf6+tY5fjXjFrPeOA356FZl3Nl4iuOW25uXO+FtGZ8Gtp/6StA3C6nhpyK+NbCe0KenuzWaHm7ZchVmMeDTtB/DfWGO19f+Fbc43fSrV3W2LGzfxmlgi9T0r+uo+8mn5zruY0nrqMnUaHFNOYwe4VuKviU9LfV06o0jL5bN9BBj1MXnqxPdXgiP5vG2PouzP3K/qtij8ddTPDJJpaZCPGX/IjeO+ejSNZ59fdfz3U0s8falFre8R7ekxX1kPOntVx/tYL+d2c/4sfDTYXfWwKPvIuOdff2dY7pxOsh62Ed24+mmX53WPBu31/uuboGX/O3KTD3R7bVw2vlqQtd/vd3HXSLLtbY2ut24XuGWVMZBfEQt8kfjjBncIzsuituo6xF8F8KjnPal+Ci5eyjjFBiX4iSVmgrxlP2L3Dhmxi3Gfrh9TMv4O1KPdnAEWO+2mceSzT3gJ8Pfjm+fAreO3cL49DBuCVzA9sv/A4TfGb5bsagTXU39VYzxjvH0r7f7uEu02N9y1Poy8SVcn9q4g51HN66oJb2O8Whkx+105wZrOaN9QOMO7vVgd/fltAy4PsV/tqipEE+xf/ed3qant7PpVxdX1NU4V2d8tMPVRkuM2/HXp1gUZ10KXwVGqaPsItvijL6j3GfMCH97ruMH9xp4+vl2Hi3j326xu/s2fotpMS1LZSz70+m+ndNx4zrnbVrjCnxd3NMsJzrtsPjr2nSaYRxncep1BRbjW0433WJPeFuZ2/M+4nZ1vQ9TPBbtNtJ6/PUUG0deHz5uXFd7nP3pCIsxbwe8TTLucFvGtdFVzumMLzZOR7gd8MT4JTUV4jGLFVaLxdVrOYSG0lLfDf+Zsz/6HnbcePXtc9znaqj1DvbvXlc7fAzHxS5GptOHX0/j49d0n8VRlgvzdpCrjfuaHFBKGcF3n9wN1FSIp+AvdCOOWzmN+5HSUt8N/5mzW56BxBqVLg/8ej5NWHS1FIpqh1LKCL67mgoRAP5Cz4PSvbRU6fA7tBX3oXLlgV9bR1PxxbdqbmqlNUIpZQTfXU2FCAB/oedB6V5aqnT4HdqK+1C58sCvLX7CECg1KaWM4LurqRAB4C/0PCjdS0uVDr9DW3EfKlce+LXFTxgCpSallBF8dzUVIgD8hZ4HpXtpqdLhd2gr7kPlygO/tvgJQ6DUpJQygu+upkIEgL/Q86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3dVUiADwF3oelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL67mgoRAP5Cz4PSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juaipEAPgLPQ9K99JSpcPvcCueXZlala+Vthb4td1PiO/4LhLyKZRSRvDd1VSIAPAXeh6U7qWlSoffIbupmB5+3Fir8rXS1gK/tmoq6kIpZQTfXU2FCAB/oedB6V5aqnT4HU7P9z+sNx63Xw1lHPC453p8BNDyMIFfW8vF8l4uZnzHd4WJcEApZQTfXU2FCAB/oedB6V5aqnT4Ha7Ef7ZP2wbfxulJj09a66EQgApDBn5tLRdLocV8RYmQT6GUMoLvrqZCBIC/0POgdC8tVTr8Drc/Uv1AUzE9CvOHu1BhyMCvbeDFgkyJkE+hlDKC766mQgSAv9DzoHQvLVU6/A6W5/uojb6TQgEbjAD82gZeLMiUCPkUSikj+O5qKkQA+As9D0r30lKlw++Q0VRc/ezWd1IoYIMRgF9bNRV1oZQygu+upkIEgL/Q86B0Ly1VOvwOlt/omP4a0mLje3iQmnzPuNhzkQQBqDBk4NfWcrFMd8ZczFeUCPkUSikj+O5qKkQA+As9D0r30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTfXU2FCAB/oedB6V5aqnT4HdqK+1C58sCvLX7CECg1KaWM4LurqRAB4C/0PCjdS0uVDr9DW3EfKlce+LXFTxgCpSallBF8dzUVIgD8hZ4HpXtpqdLhd2gr7kPlygO/tvgJQ6DUpJQygu+upkIEgL/Q86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3dVUiADwF3oelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL67mgoRAP5Cz4PSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juaipEAPgLPQ9K99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE311NhQgAf6HnQeleWqp0+B1ixenLSC/4RfBre5Xw88lTz4g/EQ4opYzgu6upEAHgL/Q8KN1LS5UOv8MXm4qKNa+YuQr4tT0mvPr8+SS1Bv8WlFJG8N3VVIgA8Bd6HpTupaVKh9/h9Gz0w7hx+terAxePXyMZXknUSlsL/NqOV8HfLT//e3VdXG2Z7nD12XLGEPAnwgGllBF8dzUVIgD8hZ4HpXtpqdLhd7h9oPHtcLtx/GsJKmauAn5t10/8p+0ZV9OpGxkPCQF/IhxQShnBd1dTIQLAX+h5ULqXliodfgc1FY+omLkK+LU1vlJ4utF+1O04IeBPhANKKSP47moqRAD4Cz0PSvfSUqXD76Cm4hEVM1cBv7bfairew286qal4CqWUEXx3NRUiAPyFngele2mp0uF3yG4qFr+nUbHmFTNXAb+2n2wq1r/gpKbiKZRSRvDd1VSIAPAXeh6U7qWlSoffYfxp6OJnosd9phunP1udnuK0TxVqpa0Ffm2ny/69bBXcV9N6KDUVT6GUMoLvrqZCBIC/0POgdC8tVTr8DqlPJ3yoRHng1/ZbCT98keJPhANKKSP47moqRAD4Cz0PSvfSUqXD76Cm4hEqUR74tf1iUzF9p5F3ug+c5cNQShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/ju86biJYQQQgghhBBmfv3+c/yjNxXiMS/47jkPSvfSUqXD79BW3IfKlQd+bfEThkCpSSllBN99TKimQjwGf6HnQeleWqp0+B3aivtQufLAry1+whAoNSmljOC7q6kQAeAv9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHc1FSIA/IWeB6V7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvvmoqTv/2YnPc2P0/M5Qw0rnmlO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL77/ZsKn8MXmwrxeZpM37S7ru5eWqp0+HDaivtQufLAry1+wqf0uRlSSo3gT6gx4U1TcfXu4rRx/LAe4fT5agf7xsUOCwvho0MZf9bSdHtRSkuVDp9BW3EfKlce+LXFT/iIVjdDSqkT+BNqT/jgTcW6c7g6dtoAXJ3luMO0Y1m3MesdcKanOn0qiXyRuyktVTp8LG3FfahceeDXFj+hgyY3Q0qpKfgTGtNUjD/mv3oVcBvi6vl+eor1UeuN0x3QpqcufSqJf5E7KC1VOnwsbcV9qFx54NcWP6GDJjdDSqkp+BMa0FRcPbK/Z68FfG8qNvuH26FOUcU+fYqJf5E7KC1VOnwsbcV9qFx54NcWP6GDJjdDSqkp+BOa21S8hyd7zKYCbVYI6FNS/IvcQWmp0uFjaSvuQ+XKA7+2+AkdNLkZUkpNwZ/QgKbiff3rT8Yn+PHwcbfpKdaDO95UoE1PXfpUEv8id1BaqnT4WNqK+1C58sCvLX5CB01uhpRSU/An1NNUfCDEh1m/bBEOOpTx9S/H7V9MtUlpqdLhM2gr7kPlygO/tvgJH9HqZkgpdQJ/Qu0JWzQVelMRS+dKUrqXliodfoe24j5Urjzwa4ufMARKTUopI/juX2gqBB/4Cz0PSvfSUqXD79BW3IfKlQd+bfEThkCpSSllBN9dTYUIAH+h50HpXlqqdPgd2or7ULnywK8tfsIQKDUppYzgu8+bipcQQgghhBBCmPn1+8/xj95UiMe84LvnPCjdS0uVDr9DW3EfKlce+LXFTxgCpSallBF89zGhmgrxGPyFngele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC766mQgSAv9DzoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDd1VSIAPAXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvvuqqTj924vNcXd2+/BQ4imdi0/pXlqqdPgd2or7ULnywK8tfsIQKDUppYzgu9+/qfA5fKapwK/vB0AoAkKGb1Hdffojg9JSpcPv0Fbch8qVB35t8RM+he9OfgWl1EjRCX3cVFy9uzhtHD+sBznudtz4s8/0kNMOizw/O1y9dTF6GbdYdjvlMY5w2vMq9of57tm/S2n36bUz3VKI0uF3aCvuQ+XKA7+2+AkfQXknv4JS6kTdCZ1ktr+pOD0fj1+6enxfj7AY1nLUYrTbkW8DrLc8PXDc/7Yg6z2/BUKGb0HgXvHOtaB0+B3aivtQufLAry1+Qgdkd/IrKKWmVJxQT1Mx/nT86qfst6d81FRMz7se9vYU+16vf3l04CLk1bBrr2+BkOFbELhXvHMtKB1+h7biPlSuPPBri5/QAdmd/ApKqSkVJ/RxU7F4NB8ff6+e3cfPj9qAjKbC6DVuWcyx/cBpUzEdcHHIF0HI8C0I3CveuRaUDr9DW3EfKlce+LXFT+iA7E5+BaXUlIoTGtlUvIcnXceT8e1j9/SonZ2ffnUx7IjxQDUVdSFwr3jnWlA6/A5txX2oXHng1xY/oQOyO/kVlFJTKk7o46biff1rQvZH3nGEn42LY41H3Uayp7o63W0wy4G35ZoOe3X4d5fad8/+XQjcR4XSUqXD79BW3IfKlQd+bfETOiC7k19BKTWl4oROMus/fieegr/Q8yjt/vqX4/YvptqkdPgd2or7ULnywK8tfsJHUN7Jr6CUOlF3QtVUiADwF3oelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL67mgoRAP5Cz4PSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/ju86biJYQQQgghhBBmfv3+c/yjNxXiMS/47jkPSvfSUqXD79BW3IfKlQd+bfEThkCpSSllBN99TKimQjwGf6HnQeleWqp0+B3aivtQufLAry1+whAoNSmljOC7q6kQAeAv9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHc1FSIA/IWeB6V7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvft9U/P2HF7HnuNoYRUbgzQHxl8IO3HZrKN1LS5UOv0NbcR8qVx74tcVPGAKlJqWUEXz3m6bi58s7Jp+pwvEs2Wd8ND7+Iting+MV1d2nHW8VqdLhw2kr7kPlygO/tvgJn9LnZkgpZQTf3dpUjH/9+/nnf8c9fzZO95kOdTXI9FI5bj/taUl1teU225hqMewY7OqM6wGvNoKAmeozlHY/Xi/jdnBKh8+grbgPlSsP/NriJ3xEq5shpZQRfPfJIhx//Wnx+fQoPB30dp/1DuvrxDHgenD3UJYdpmcZy2XMiQN4vFQI3Et/KyodPpa24j5Urjzwa4uf0EGTmyGllBF89/um4j17zH30pP7osTuqqZjucORq8EfZLMNOW6xFhqsBkRcTcrZsCNxLfysqHT6WtuI+VK488GuLn9BBk5shpZQRfHdTU/Eenqc/0FS8r5+kN7uUq6EsedY72EdYBFssGtjWAjPVZyBwL/2tqHT4WNqK+1C58sCvLX5CB01uhpRSRvDdb5qKpz3AdNDAHiB8wFpNxe1XvwVmqs9A4F76W1Hp8LG0FfehcuWBX1v8hA6a3AwppYzgu9+/qRh/Om5/yXB6rfE2PJdfDTLmngZbnPF4lL3TWA9lGXYa8hR+PPxqy5gWAdhgH4DAvfS3otLhY2kr7kPlygO/tvgJHTS5GVJKGcF3v28qbg9IYvFcLtDoPEGl3V//ctz+xVRGSofPoK24D5UrD/za4id8RKubIaWUEXx36KYC/Mfz4ofOc0TpXlqqdPgd2or7ULnywK8tfsIQKDUppYzguz9uKoQYwV/oeVC6l5YqHX6HtuI+VK488GuLnzAESk1KKSP47moqRAD4Cz0PSvfSUqXD79BW3IfKlQd+bfEThkCpSSllBN993lS8hBBCCCGEEMLMr99/jn/0pkI85gXfPedB6V5aqnT4HdqK+1C58sCvLX7CECg1KaWM4LuPCdVUiMfgL/Q8KN1LS5UOv0NbcR8qVx74tcVPGAKlJqWUEXx3NRUiAPyFngele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC766mQgSAv9DzoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDdb5qK47+9+HwUUYXOc0fpXlqqdPgd2or7ULnywK8tfsIQKDUppYzgu983FYtdM04vKtJkHqfddXX30lKlw4fTVtyHypUHfm3xEz6lz82QUsoIvruzqRhfX1y901jvefzqeK7FaPiVbUWH6RhX6XF7UUpLlQ6fQVtxHypXHvi1xU/4iFY3Q0opI/junqYiY+P4edx/OoJAoM+MUN61S0uVDh9LW3EfKlce+LXFT+igyc2QUsoIvvt9U3H7qmEcKKSpWByIX9Zu9JkRyrt2aanS4WNpK+5D5coDv7b4CR00uRlSShnBd7e+qbh64p9uTGoq3he/NSi+Tp9Jobxrl5YqHT6WtuI+VK488GuLn9BBk5shpZQRfPfdX3+y7xn4pkKg0WdqKO/apaVKh4+lrbgPlSsP/NriJ3TQ5GZIKWUE393TVLxnvxY1/UWp4/arU/581f6mAr+s3egzI5R37dJSpcPH0lbch8qVB35t8RM6aHIzpJQygu9+01TsDBTO7a9giW/RYTpe/3Lc/sVUm5SWKh0+g7biPlSuPPBri5/wEa1uhpRSRvDdizUVelOBSecZoXQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDdw5oK0Rn8hZ4HpXtpqdLhd2gr7kPlygO/tvgJQ6DUpJQygu+upkIEgL/Q86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3edNxUsIIYQQQgghzPz6/ef4R28qxGNe8N1zHpTupaVKh9+hrbgPlSsP/NriJwyBUpNSygi++5hQTYV4DP5Cz4PSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juaipEAPgLPQ9K99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE3/2mqTj+2wvjKDvO+PV6/7cm306BReeCULqXliodfoe24j5Urjzwa4ufMARKTUopI/ju903F01HwnXfw2XHX5N1A8C/TfrK6e2mp0uHDaSt+i9bJh8GvLX7Cp/RZ5JRSRvDdPU3F+O5i0VScdl4fNd1yNeC4ffx8+6blyuXq+hx1xsOv9rlynFZpIT4V+S6YqWIZp++4vSilpUqHz6Ct+Bqtk8+DX1v8hI9otcgppYzgu08W4frXn66eiacjTh/W7V8dx5+e0RFsfcjTs0z3tMTw7QAIeLxAKO/apaVKh4+lrbgFrZNPgl9b/IQOmixySikj+O73TcXiAEdT8fNh/F/7sfvBog4Zm67T9qdeV8OeToEGcrZYKO/apaVKh4+lrbgFrZNPgl9b/IQOmixySikj+O7Fmor39eP1V5qKzRjGkx6/hLmkMFNlQHnXLi1VOnwsbcUtaJ18Evza4id00GSRU0oZwXf/QlOx83n9wP3zoURTYc95NT4OmKkyoLxrl5YqHT6WtuIWtE4+CX5t8RM6aLLIKaWM4Ls/bireh9/Pme52OmTcebG//U3FIthx/9eBcf+nLgu7acJx4yLneti1xdeBDRYO5V27tFTp8LG0FbegdfJJ8GuLn9BBk0VOKWUE3/2mqUBj/cS/3l/k0aHOr385bv9iqk1KS5UOn0Fb8TVaJ58Hv7b4CR/RapFTShnBd6/XVDz6mT3+BHDQuc6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvnuxpkJggr/Q86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3dVUiADwF3oelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL77vKl4CSGEEEIIIYSZX7//HP/oTYV4zAu+e86D0r20VOnwO7QV96Fy5YFfW/yEIVBqUkoZwXcfE6qpEI/BX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juaipEAPgLPQ9K99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE311NhQgAf6HnQeleWqp0+B3aivtQufLAry1+whAoNSmljOC73zcVP//24oOpnBhDTncrIQhL5+pRupeWKh1+h7biPlSuPPBri58wBEpNSikj+O43TcXxy09lsvd3j4Y/K+XoXFJK99JSpcPv0Fbch8qVB35t8RM+ZfrD3+qalFJGiro/aCqOG0+2fz8fNx7/76WmR50OH/efnu6qyTl9dX32Me3VaKcwi5DN6VwQSvfSUqXD79BW3IfKlQd+bfETPmJ8QDpuLwqllJG67o+biumD+LRbWD+yv4eqXTUwt2cfh1p8NqZd7IA/qZ+nc00o3UtLlQ6/Q1txHypXHvi1xU/ooOIz6C2UUkYquoc1Fbcbpz/mXzz6Hw8cA4z/uxjN0pkYW6CrnM3pXBNK99JSpcPv0Fbch8qVB35t8RM6qPgMegullJGK7p9rKm5PvyjW+Org803FKYY40rkslO6lpUqH36GtuA+VKw/82uIndFDxGfQWSikjFd0fNBWnJ/j33RO55S3E1f7roDufn6bVCwoLnYtD6V5aqnT4HdqK+1C58sCvLX5CBxWfQW+hlDJS0f2mqXgffnNpveXq8/Gh/Hjg1YP72ABMN95+vjr74tRGHfxJ/Tyda0LpXlqqdPgd2or7ULnywK8tfkIHFZ9Bb6GUMlLR/b6pEH+5ffHSmc4FoXQvLVU6/A5txX2oXHng1xY/4SNe/3Lc/sVUm1BKGanrrqbCynSCxV8614TSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd1dTIQLAX+h5ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juaipEAPgLPQ9K99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE333eVLyEEEIIIYQQwsyv33+Of/SmQjzmBd8950HpXlqqdPgd2or7ULnywK8tfsIQKDUppYzgu48J1VSIx+Av9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHc1FSIA/IWeB6V7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvrqZCBIC/0POgdC8tVTr8Dm3FfahceeDXFj9hCJSalFJG8N1XTcXp315sjpt07NXOHwsg3r0rRuleWqp0+B3aivtQufLAry1+whAoNSmljOC737+p8Dl8xVyNxLfoXL3q7tMfGZSWKh1+h7bit/AtcnDwa4ufMITqmp2v3KLuj5uKq3cXp43jh/UIxy3rka92Pg17tf80wOLYqwDviynvSec6lHafLvjplkKUDr9DW/E1lIscHPza4icMobRm5yu3rvvjpmL6pSvz6bHj0/nVxpCd1yLuAAv3hnQuAoF7xTvXgtLhd2grboFskYODX1v8hCEQaHa+ciu6e5qKq5/0j/vcnvJjTcXVy4fYAG3pXAQC94p3rgWlw+/QVtwC2SIHB7+2+AlDINDsfOVWdH/cVKwf3E+P3ThvKmLHVFNxonMRCNwr3rkWlA6/Q1txC2SLHBz82uInDIFAs/OVW9E9sql4D4/dIE2FRUFNxQ6di0DgXvHOtaB0+B3ailsgW+Tg4NcWP2EIBJqdr9yK7o+bivf1rz/dPn9fHf5ePg7D06UAACAASURBVKY7dh73GUe42rgZoCedi0DgXvHOtaB0+B3ailsgW+Tg4NcWP2EIBJqdr9yK7vdNxQdOKarTeU5Lu7/+5bj9i6k2KR1+h7biaygXOTj4tcVPGEJpzc5Xbl13NRUigM5zSuleWqp0+B3aivtQufLAry1+whAoNSmljOC7f6GpEHzgL/Q8KN1LS5UOv0NbcR8qVx74tcVPGAKlJqWUEXx3NRUiAPyFngele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC7z5vKl5CCCGEEEIIYebX7z/HP3pTIR7zgu+e86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3ceEairEY/AXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvruaChEA/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3V1MhAsBf6HlQupeWKh1+h7biPlSuPPBri58wBEpNSikj+O43TcXx3164R7zamMfidPhTUpHOVaV0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3e+bisWuxhHtx0aBX3cyOhe8uvv0RwZVpEqH36GteCAqVx5QtW1ysTTRfJNKGYFyNy65m6bi9OJifI8xfblxOs3VDj8bF+9GFmccTzc9xdVJ1wpiQedClXYfL4rjdnBKh9+hrXgsKlceOLVtcrE00fwLpZQRHHf7krtvKhZffbTxtMO4synu8nS3+ReH4EwePp1rReBe+ltR6fA7tBWPQuXKA622TS6Wtpp9QHP3NBWnn9mHNBXGYadxx7cKlgyO/GiTh0znWhG4l/5WVDr8Dm3Fo1C58kCrbZOLpa1mH9Dct95UjFv231TYhzrt/PR09qbiNL6w0LlcBO6lvxWVDr9DW/EoVK480Grb5GJpq9kHNHeSpuLR6SyZjScVV3QuGoF76W9FpcPv0FY8CpUrD7TaNrlY2mr2Ac09uKl4H36XadyyaCTGfYzP9+PIVxmuMtvfVKBNHjKda0XgXvpbUenwO7QVj0LlygOttk0ulraafUBzf9xUNOT2pYoY6Vyo0u6vfzlu/2IqI6XD79BWPBaVKw+c2ja5WJpo/oVSygiOu33JqamYlEms6VwrSvfSUqXD79BW3IfKlQd+bfEThkCpSSllBN9dTYUIAH+h50HpXlqqdPgd2or7ULnywK8tfsIQKDUppYzgu6upEAHgL/Q8KN1LS5UOv0NbcR8qVx74tcVPGAKlJqWUEXz3eVPxEkIIIYQQQggzv37/Of7RmwrxmBd895wHpXtpqdLhd2gr7kPlygO/tvgJQ6DUpJQygu8+JlRTIR6Dv9DzoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDd1VSIAPAXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvruaChEA/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3v28q/v7Di/Az/Qy7M3hgffGnaodHdo5ScFdvDaV7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvftNU/Hw51gSkkfjAsA4ykmTb4VTv81C6l5YqHX6HtuK3TH80pnKFgF9b/IQhNNF8k0qN4E+oMeG8qTiNMo51u/HYn0y3Tw85DWs/9ph/ar7QsbucvI7nspRo3PO2vLd2iz0XyU8Z1rFH1l/lhtK9tFTp8Du0FV8z/WYx3SKegl9b/IQhNNH8C6XUCfwJtSec/PrT1V9vH1VvD1x8vgr6aJzFs3K4y/Rclqf59blGjHbGGMayGCM1hNK9tFTp8Du0FbeA/O25Ovi1xU8YQltNVvAn1NNUvK+fMscfda9/4P3o8ziyY5zbwY2H7HdWtyW6bSoeleJ2z2lncjXUNM/VId2gdC8tVTr8Dm3FLeB/e64Lfm3xE4bQVpMV/Al1NhXvi0fe9XPwflPxqE/AbyreA75i2qNamoqfjYsAt60F2kL/JJTupaVKh9+hrbgF/G/PdcGvLX7CENpqsoI/oY+bivXDsZqK9duAxQj2c+1EtTRFjgBPv8oNpXtpqdLhd2grbgH/23Nd8GuLnzCEtpqs4E/o46biPfyU+nVg3O1q5/fFw+6Y4DTIYtjbY9cnMsZeBzsOa3nEHwe5ijqdqkeHX+15tbPxq1NudyCG0r20VOnwO7QVtzC9o34lCR/4tcVPGEJbTVbwJ9SS8OY/foemJJ6SMYOdVwWle2mp0uF3aCu+5vUvx+1fTMUBfm3xE4bQRPMvlFIn8CfUnlBNBTlqKmKhdC8tVTr8Dm3FfahceeDXFj9hCJSalFJG8N0fNxVCjOAv9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHc1FSIA/IWeB6V7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvPm8qXkIIIYQQQghh5tfvP8c/elMhHvOC757zoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDdx4RqKsRj8Bd6HpTupaVKh9+hrbgPlSsP/NriJwyBUpNSygi+u5oKEQD+Qs+D0r20VOnwO7QV96Fy5YFfW/yEIVBqUkoZwXdXUyECwF/oeVC6l5YqHX6HtuI+VK488GuLnzAESk1KKSP47qum4vS19V+nW9bbv85OsM9IwZbuRJWcGVC6l5YqHX6HtuI+VK488GuLnzAESk1KKSP47pFNhf2rBNh7qkeD7I+ZRPPpXkDpXlqqdPgd2or7ULlC+Pv/9zJu/EoYO/gJn1J0IhxQShnBd7/59aefL//9cPrr+7/rePzqdPtxnLFFOR4y/dLVlvF000PG8y7GnG5Z5By3r3czKtvzjDpT8fUIV7W6tRsV+kDpXlqqdPgd2or7ULn2Gb/1HLcjg5/wEXUnwgGllBF898kifNpUTMdafF4Msr4k1ied1vr0EGw/nf2866jG092OYxnwah9jwe21anLnMkLpXlqqdPgd2or7ULmiqPgdAT+hg4oT4YBSygi+u6mpmP4Yezze/Tm8qZj+ZH19yGIEX1MxPXxxxk1l34F2zdHuVqQJlO6lpUqH36GtuA+VK4qKz7L4CR1UnAgHlFJG8N0hmor37Dl1sf/4RGvvW9bjTD8bd7g9fEyL0FQ4AhhFmkDpXlqqdPgd2or7ULmiqPgsi5/QQcWJcEApZQTf/aapmO8R10hcncK+vzHe03HymoqrAJv5T9s/0FTcfrUJlO6lpUqH36GtuA+VK4qKz7L4CR1UnAgHlFJG8N2zmor34dHW/qbi6sn19KXpiaajPWoqbge52jgqX2UYD5wOtT71wn2suV18fchp8PWMtILSvbRU6fA7tBX3oXJFMf0W9pUkdvATOqg4EQ4opYzgu08W4ef/43c7j6d6tEWgc+Up3UtLlQ6/Q1txHyrXPq9/OW7/YioL+AkfUXciHFBKGcF3R2kqpj+Gzz5WRNG5+JTupaVKh9+hrbgPlSsP/NriJwyBUpNSygi+O0RTIaqDv9DzoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDd1VSIAPAXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvvu8qXgJIYQQQgghhJlfv/8c/+hNhXjMC757zoPSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBdx8TqqkQj8Ff6HlQupeWKh1+h7biPlSuPPBri58wBEpNSikj+O5qKkQA+As9D0r30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTfXU2FCAB/oedB6V5aqnT4HdqK+1C58sCvLX7CECg1KaWM4LvfNBXHf3tx2jh+3jyxqEvn2aR0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3e+bisUBT/VCyuEYBH8aqtO5wpTupaVKh9+hrfgt0x9+qVwh4NcWP2EITTTfpFIj+BNqTLhqKl4XnL56e8hx5GmXMh3qNJrlpNNDprtN/2oJZjmXMXZdaEQcULqXliodfoe24mtO33FO28UO+LXFTxhCE82/UEqdwJ9Qe8KbX3+aPmpffR6frdf7jCnXiS0nnW6xZDMGu5rjcYfbGtaFw8IHpXtpqdLhd2grbgH523N18GuLnzCEtpqs4E+op6lYHDD9gf1VB7I+9v3vM/c41KOTLs57tXHRMKyDjeNc7WAUrAiHhQ9K99JSpcPv0FbcAv6357rg1xY/YQhtNVnBn9DcpuL2fPam4tLA/GP+xXO/PZslmKUgp+YBbVnsw2dkh9K9tFTp8Du0FbeA/+25Lvi1xU8YQltNVvAnFLSpsAy1s+d+w2McZ9GZLE5NAJ+RHUr30lKlw+/QVtxCh/vwt8CvLX7CENpqsoI/oZ6m4vRbOuvG4LTzcePxr9Njb4c6fXWRcDxwOpTxwHUwRxFuBcvBYeGD0r20VOnwO7QVtzC9/38lCR/4tcVPGEJbTVbwJ9SSUP/xu2BOvcQXk+TB6mWB0r20VOnwO7QVX/P6l+P2L6biAL+2+AlDaKL5F0qpE/gTak+opiIYspcSU4jVbqF0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3dVUiADwF3oelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL67mgoRAP5Cz4PSvbRU6fA7tBX3oXLlgV9b/IQhUGpSShnBd583FS8hhBBCCCGEMPPr95/jH72pEI95wXfPeVC6l5YqHX6HtuI+VK488GuLnzAESk1KKSP47mNCNRXiMfgLPQ9K99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE311NhQgAf6HnQeleWqp0+B3aivtQufLAry1+whAoNSmljOC7q6kQAeAv9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHdTU/Hzzy/CT/YxHp36aued/PhLYQduuzWU7qWlSoffoa24D5UrD/za4icMgVKTUsoIvvt9U3HcY9NncrLnA364pmokLPQxHanuPv15QWmp0uF3aCt+C98ixwG/tvgJ86iu2XzuKrrfNBULgenri/XG2+23Y9qHGg9c55zufBr2av+1iz3A+2IZ4VMxcxSl3aeLc7qlEKXD79BWfA3lIgcBv7b4CVMprdl57uq6TzJbmorxafjRxvU4V4cvIj069f7OxjBPAyyswSkXOBAC94p3rgWlw+/QVtwC2SKHAr+2+AmTINBsO3fvmu7WpmL8Mfx0H+PG6ZbXvxw3rhM/PXVUUzH+1VKlpwFKUC5wIATuFe9cC0qH36GtuAWyRQ4Ffm3xEyZBoNl27t413a1NxemvSU3FIuX6p/hfbyryApSgXOBACNwr3rkWlA6/Q1txC2SLHAr82uInTIJAs+3cvWu63zQV7ydPwHlNxdWAjpAhO48x1FR8O8LXIHCveOdaUDr8Dm3FLZAtcijwa4ufMAkCzbZz967pft9UvA+/2LO58aofOD5Yn/ZcDHh7inUP8HTncZ9xhKuNmwHwKRc4EAL3ineuBaXD79BW3ALZIocCv7b4CZMg0Gw7d++a7pPMFf/jd/iF5qZz/Uu7v/7luP2LqTYpHX6HtuJrKBc5CPi1xU+YSmnNznNX111NhQigc/0p3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfHeSpkJ8F/yFngele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC766mQgSAv9DzoHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDd503FSwghhBBCCCHM/Pr95/hHbyrEY17w3XMelO6lpUqH36GtuA+VKw/82uInDIFSk1LKCL77mFBNhXgM/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3V1MhAsBf6HlQupeWKh1+h7biPlSuPPBri58wBEpNSikj+O5qKkQA+As9D0r30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTf/aap+PmHF7HnEGR0nmJK99JSpcPv0Fbch8qVB35t8ROGQKlJKWUE3/2+qbjab+ccnzxcfIDOc0TpXlqqdPgd2or7ULlCmP7YEaq2+AlDoNSklNoByt04O/dNxeLz9M3G37+etk8/j3uOY46nCHmdInboXHxK99JSpcPv0Fbch8q1z8+37Ol2BPAThkCpSSm1CY67fXbmv/5k7AeuzrHe7dQwWPZcn1F8mM5TQOleWqp0+B3aivtQuaLAf+zDTxgCpSallBs0d/+birehH7CcwzjIYuP03cVbfJXOU0DpXlqqdPgd2or7ULmiwH/sw08YAqUmpZQbNPfEpuJ98ftVjwZ5dLr1RvFJOk8BpXtpqdLhd2gr7kPligL/sQ8/YQiUmpRSbtDcg5sK4+sC3yBqKgrReQoo3UtLlQ6/Q1txHypXFPiPffgJQ6DUpJRyg+buaSpOv2t03P4envinLytu32w8egcyRkKrckM6TwGle2mp0uF3aCvuQ+WKAv+xDz9hCJSalFJu0NwfNxW+odG0RTadZ5zSvbRU6fA7tBX3oXLt8/qX4/YvpjqCnzAESk1KqU1w3O2z428qxtFFEzpPOqV7aanS4XdoK+5D5coDv7b4CUOg1KSUMoLvHtZUiM7gL/Q8KN1LS5UOv0NbcR8qVx74tcVPGAKlJqWUEXx3NRUiAPyFngele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC7z5vKl5CCCGEEEIIYebX7z/HP3pTIR7zgu+e86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3ceEairEY/AXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvruaChEA/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3V1MhAsBf6HlQupeWKh1+h7biPlSuPPBri58wBEpNSikj+O73TcXxn1984PQ7x+KXm5XOlad0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3W+aitOXoXzUVODQufKU7qWlSoffoa24D5UrhOkPHPFri5/wKUUnwgGllBF8d2dTcXp3cXqbMX25cfzSdPzpIeN1snhzsjjv4nRXUsJO56JRupeWKh1+h7biPlSufX6+rU+3I4Of8BF1J8IBpZQRfHd/U7HeOP3quOfiGrg9xfr6MY6zDi+MdK4epXtpqdLhd2gr7kPliqLisyx+QgcVJ8IBpZQRfPespmL6HmB9yGKEzzQV+LMFS+fSUbqXliodfoe24j5UrigqPsviJ3RQcSIcUEoZwXcPaCred7/UZNx4tY99h0cvN6ZdhFoLH52LRuleWqp0+B3aivtQuaKo+CyLn9BBxYlwQCllBN898k3FdMujlxvrZJvnvXrjsRhEGOlcNEr30lKlw+/QVtyHyhVFxWdZ/IQOKk6EA0opI/juN03Fe/aLTONh0x1OGy1Nxe0gVxvXX/rZsjjd1ZjCQue6UbqXliodfoe24j5UrigqPsviJ3RQcSIcUEoZwXe/byoeDYEv/JeKmZHpXENK99JSpcPv0Fbch8q1z+tfjtu/mMoCfsJH1J0IB5RSRvDdY5qKcj/mr5gZmc5lpHQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDdA5oKIfAXeh6U7qWlSoffoa24D5UrD/za4icMgVKTUsoIvruaChEA/kLPg9K9tFTp8Du0FfehcuWBX1v8hCFQalJKGcF3nzcVLyGEEEIIIYQw8+v3n+MfvakQj3nBd895ULqXliodfoe24j5Urjzwa4ufMARKTUopI/juY0I1FeIx+As9D0r30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTfXU2FCAB/oedB6V5aqnT4HdqK+1C58sCvLX7CECg1KaWM4LurqRAB4C/0PCjdS0uVDr9DW3EfKlce+LXFTxgCpSallBF891VTcfq3F1cHpKaxf/UzPMqAEPgz9DEdoXQvLVU6/A5txX2oXHng1xY/YQiUmpRSRvDd799UnPZQU/HtCIh0Lkt19+OPDI4bvxLmKaXDh9NW3IfKFQL+NYifMIQmmm9SqRH8CTUmNDUVp7HGtxnjDlefjztPB7/a+dHIlkGupKanuNp4qsP01KNadSiljJR2P15343ZwSofPoK24D5VrH/xrED9hCE00/0IpdQJ/Qu0J75uK8aH59il/vfH0UH6ZbGPkKcbz3o5sKS7OUsiA224NgTvyneuW0uFjaSvuQ+WKAv8axE8YQltNVvAnNKapGD8HNhXGrDtNxfgKYpH/ds9pZ3I11BiGA2K1Wwjc8e9cC0qHj6WtuA+VKwr8axA/YQhtNVnBn1CspuI9PGpHNRXjyI5BjBvH+ixaGtbWglLKCIE7/p1rQenwsbQV96FyRYF/DeInDKGtJiv4E/r9puL2mduY9enIjvy3G69OtI6BtiZCoJQyQuCOf+daUDp8LG3FfahcUeBfg/gJQ2iryQr+hH6iqXgffmto3DiOcNUMXP2Y3zfyepxFU7TY8/aMxjwEsHpZIHDHv3MtKB0+lrbiPlSuKPCvQfyEIbTVZAV/Qj1NxWfSxFYqb+SrszSncylKu7/+5bj9i6mMlA6fQVtxHyrXPvjXIH7CEJpo/oVS6gT+hNoTfrSpGAOBj3w6S97gtehcCkr30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTf/ZtNhaABf6HnQeleWqp0+B3aivtQufLAry1+whAoNSmljOC7q6kQAeAv9Dwo3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfPd5U/ESQgghhBBCCDO/fv85/tGbCvGYF3z3nAele2mp0uF3aCvuQ+XKA7+2+AlDoNSklDKC7z4mVFMhHoO/0POgdC8tVTr8Dm3FfahceeDXFj9hCJSalFJG8N3VVIgA8Bd6HpTupaVKh9+hrbgPlSsP/NriJwyBUpNSygi+u5oKEQD+Qs+D0r20VOnwO7QV96Fy5YFfW/yEIVBqUkoZwXdXUyECwF/oeVC6l5YqHX6HtuI+VK488GuLnzAESk1KKSP47jdNxc+/5v5wCMfOOyHx5wmczgWs7j69wKtIlQ4fTlvxW7ROPgx+bfEThlBdU1fuCXz3+6biaj8E1EiA0LmYpd3/hi961y4dPoO24mu0Tj4Pfm3xE4ZQWlNX7gi+u6epWHyevtmYbj9uudp/vfNp2Kv9pwEWx14F+Nn+FgOdy0LgXvquXTp8LG3FLWidfBL82uInDIFAU1fuEXz3+6bi6gn79Hnaflj2t3x+tPNiy06AhaDoXBMC99J37dLhY2krbkHr5JPg1xY/YQgEmrpyj+C73zcV4+cSTcXVy4fYAOIvnWtC4F76rl06fCxtxS1onXwS/NriJwyBQFNX7hF897Cm4n3920FfbyryAoi/dK4JgXvpu3bp8LG0FbegdfJJ8GuLnzAEAk1duUfw3beaCuMvBX2+qbA0GGoqAulcEwL30nft0uFjaStuQevkk+DXFj9hCASaunKP4LvfNxWn3xo6bn8PT9vTlxXTQdY9wNOdb5NcZRiPfRpAvHvXpLT761+O27+Yykjp8Bm0FV+jdfJ58GuLnzCE0pq6ckfw3SctgO8/frd+7ncMKArReYop3UtLlQ6/Q1txHypXHvi1xU8YAqUmpZQRfPfIpmJsKK/OIcjoPMWU7qWlSoffoa24D5UrD/za4icMgVKTUsoIvntYUyE6g7/Q86B0Ly1VOvwObcV9qFx54NcWP2EIlJqUUkbw3edNxUsIIYQQQgghnqCmQgghhBBCCLHFr99/fv6oqRBCCCGEEEI8ZvKmIv0XrwQRvRfMi/RPVX79/vPtCN+hrbgPlSuPCrX9+g1Wd3I3Xy+pJnSOmgoRQ+8F8/VbTLs715oKDzQptBX3oXLlUaG2X7/B6k7u5usl1YTOUVMhYui9YL5+i2l351pT4YEmhbbiPlSuPCrU9us3WN3J3Xy9pJrQOWoqRAy9F8zXbzHt7lxrKjzQpNBW3IfKlUeF2n79Bqs7uZuvl1QTOuemqVj8wwvHmbaSCmx6z+/XbzHt7lxrKjzQpNBW3IfKlUeF2n79Bqs7uZuvl1QTOsf0psLyvHi7T++HTn6azO9Fg/31W8yDP//Nf95y2gh+5/rL1U89KjzQ7DL9QU8HcTc910k2Va7B2a37+3dj3cnXXP9c+/v1/9gUg0/oaXacTcXVKK/h/cZiEMFEh/m9Xs/fv/vYb1I//3vcMn5Gu3PdgvxAE87Pnfa0nV7czfQGpXLFAnsNXnx7+v4NWXfyNdfPFd+fgs9MMfiE3nYH/9m4biqmX7qa+9sdBAfd5rdoU7G+W8HeuW45LT+oB5o81FQYubo7qVyBwF6DxA+ms59h176TnyCeu0eTizmh09nJairGlxXdHjq70WR+pxfM1+8+UXcrzDuXBdgHmlTUVBiZvjx/q1yhwNb2ava/fh+OvZMfRYc9q0I8d8b5RZ7Q6eykNBWWrkOQ0W1+//X9/t3n6a3q9jPOneuWts/WbcWfcvXtSeWKAnkpXj/MfP9WHHgnX25ngGzuHk0u5oRePeqrqRABdJvf0jc4NRUctBV/ipqKbJCXopqKz9c8A7K5ezS5mBMa1lRcHnbYMr4W6fbQ2Y0O80vzzYmpqZguPJwHmlSQn+SgUFORCvg1SHPf5r6TTyGeO/fk4kyov6kQ4pYmC2Z6wXz97vPoPvVvw//PxmH/AoA/0CQxzON/oBd3M71yVa4Q8K/B6vftDnfyK/jmzjHLsBM6zo6aChFD7wXz/btPzp+qQD3QfJK24j5Urjwq1PbrN1jdyd18vaSa0DlqKkQMvRfM128x7e5cayo80KTQVtyHypVHhdp+/QarO7mbr5dUEzpHTYWIofeC+fotpt2da02FB5oU2or7ULnyqFDbr99gdSd38/WSakLnrJoKIYQQQgghhLCjpkIIIYQQQgixxa/ff37+qKkQQgghhBBCPOb8puITv3sliKjwG7RZULqXliodfoe24j5Urjzwa4ufMARKTUopI/juY0I1FeIx+As9D0r30lKlw+/QVtyHypUHfm3xE4ZAqUkpZQTfXU2FCAB/oedB6V5aqnT4HdqKob9fqQAAABlJREFU+1C58sCvLX7CECg1KaWM4LuPCf8X2qAKN+yISu4AAAAASUVORK5CYII=" /><br />
<br />
Looking at the last row, we see that the three most important traits for a QFO in our organization are being Organized and Thorough, possessing a Broad Vision regarding the company's goals and being a Critical Thinker. We know that they are important because they affect the largest amount of skills we defined as important for a QFO's daily work.<br />
<br />
Next post I'll do more fine tuning and show how to use this method for evaluating employees and giving them more effective feedback.<br />
<br />
<br />
<table border="0" cellpadding="0" cellspacing="0" style="width: 1168px;"><tbody>
<tr height="40" style="height: 30.0pt;"><td class="xl67" height="40" style="height: 30.0pt; width: 166pt;" width="221"><br /></td><td align="left" class="xl68" style="border-left: none; width: 74pt;" width="99"><br /></td><td align="left" class="xl68" style="border-left: none; width: 62pt;" width="83"><br /></td><td align="left" class="xl68" style="border-left: none; width: 74pt;" width="98"><br /></td><td align="left" class="xl69" style="border-left: none; width: 89pt;" width="119"><br /></td><td align="left" class="xl68" style="border-left: none; width: 103pt;" width="137"><br /></td><td align="left" class="xl68" style="border-left: none; width: 48pt;" width="64"><br /></td><td align="left" class="xl69" style="border-left: none; width: 68pt;" width="90"><br /></td><td align="left" class="xl68" style="border-left: none; width: 63pt;" width="84"><br /></td><td align="left" class="xl68" style="border-left: none; width: 74pt;" width="99"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td><td class="xl71" style="border-left: none; border-top: none;"><br /></td></tr>
<tr height="20" style="height: 15.0pt;"><td class="xl70" height="20" style="border-top: none; height: 15.0pt;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td><td class="xl72" style="border-left: none; border-top: none;"><br /></td></tr>
</tbody></table>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com6tag:blogger.com,1999:blog-1139266731001119488.post-1324879672223435772011-10-04T13:27:00.000-07:002015-02-04T00:19:20.985-08:00The role of QA in a Continuous Development environment<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
One of the challenges we, as a QA organization, face in a Continuous Development environment, is that the "old" definitions of QA are no longer valid for us. We are not the "gate keepers". We do not give a "green light" or "Go-no-go" signals. Why? Because if we want to deploy a new version with new features on a daily basis in production, we need to move fast and repetitive dev and QA cycles are a huge overhead.<br /><br />In the following bullets I have summarized the challenges our R&D organization faces and the way we in the QA team should tackle those challenges.<br /><span style=" ;font-family:Symbol;color:#000099;" ><span style=""><span style="font:7.0pt "Times New Roman""><br /></span></span></span><ul><li><span style="color:#1F497D;">What do we want to achieve? </span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">We want to release features as soon as they are ready, regardless of other features’ readiness.</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">We want to keep the time between the <i>developer starts coding</i> and the <i>feature is deployed</i> in Prod to a minimum.</span><span style="color:#000099;"></span></p> <ul><li><span style=" ;font-family:Symbol;color:#000099;" ><span style="">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">How are we going to keep the development cycle short while releasing on a daily basis?</span><span style="color:#000099;"></span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">Reduce the amount of “links” in the release chain – only one person (FO, development Feature Owner) will be needed to take a version to Production (no dev->QA->dev->QA->OPS->Prod, but instead only dev->Prod)</span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Reduce the overhead of Dev/QA interaction. This overhead is bearable when releasing a version every two weeks (if the QFO (aka tester in some organizations) is stuck she can do something else) but becomes unbearable when releasing on a daily basis (if the QFO is stuck, no QA work is done and as result the whole process is stuck).</span><span style="color:#000099;"></span></p> <ul><li><span style=" ;font-family:Symbol;color:#000099;" ><span style="">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">How are we going to reduce the number of links in the chain and reduce dev/QA friction?</span><span style="color:#000099;"></span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">QFOs (testers) will no longer be a “required phase” for releasing the version. Instead, QFOs will now specialize in <span style="font-weight: bold;">providing tools</span> for the development Feature Owners to verify that the released version is of high quality. If the QFOs provide high quality tools and the FO uses them properly, the version will reach production in high quality.</span><span style="color:#000099;"></span></p> <ul><li><span style=" ;font-family:Symbol;color:#000099;" ><span style="">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">What tools will the QA provide for the FOs?</span><span style="color:#000099;"></span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Critical review of Spec and Design</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Test cases</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Automatic tests (where the QFO is able to code automation)</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">Exploratory tests once the feature reaches production</span></p> <ul><li><span style=" ;font-family:Symbol;color:#000099;" ><span style="">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Who does the actual testing? It depends:</span><span style="color:#000099;"></span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">Automation can be written either by the development Feature Owner or QA Feature Owner, depending on availability and proficiency in automation coding.</span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">High priority manual tests are usually performed by FO. QFO assists when needed.</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">Low priority tests are QFO domain</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">When a feature reaches production, after it is deployed, QA run exploratory tests on it, to see if we missed anything</span><span style="color:#000099;"></span></p> <ul><li><span style=" ;font-family:Symbol;color:#000099;" ><span style="">·<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Who owns the quality?</span><span style="color:#000099;"></span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">FO owns the quality of each release. The FO bases her assessments of the release quality based on the tools provided by QA.</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left:1.5in; text-indent:-.25in"><span style=" ;font-family:Wingdings;color:#000099;" ><span style="">§<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Why can’t QA own the quality of each release? Because if the QA is to own it, we will need time testing it and this is the exact overhead we’re trying to cut out.</span><span style="color:#000099;"></span></p> <p class="x_MsoListParagraph" style="margin-left: 1in; text-indent: -0.25in;"><span style="Courier New"; font-family:";color:#000099;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color: rgb(31, 73, 125);">QA owns the general quality of the application. They are required to reflect the quality of the application to decision makers and help them focus the bug fixing resources on the most painful areas.</span></p><p class="x_MsoListParagraph" style="margin-left:1.0in; text-indent:-.25in"><span style="color:#1F497D;"><br /></span></p>Our CEO, having seen the above bullets in an email thread, challenged me and asked: "so if you guys are not doing the testing and not the owners of quality, why do we need QA?". Fair question, to which I gave the following reply:<br /><br />Why do we need QA?<br /><br /><ul><li><span style=" ;font-family:Symbol;color:#1F497D;" ><span style=""><span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">(Most) developers don’t write good tests. </span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.25in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#1F497D;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Developers tend to think positively instead of negatively. The tests that they will write will make sure that the feature works where used properly, but will not find where it breaks.</span></p> <p class="x_MsoListParagraph" style="margin-left:1.25in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#1F497D;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">QFOs usually have broader view of the system and see interactions between components developers tend to neglect when designing tests.</span></p> <ul><li><span style=" ;font-family:Symbol;color:#1F497D;" ><span style=""><span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">QA still owns the quality of the application <u>end to end</u>. </span></li></ul> <p class="x_MsoListParagraph" style="margin-left:1.25in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#1F497D;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="color:#1F497D;">QA is responsible for examing all the exiting problems in the system and generating relevant heat maps that will serve decision makers when deciding resource balancing between developing new features and fixing bugs.</span></p> <p class="x_MsoListParagraph" style="margin-left:1.25in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#1F497D;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">Even if developers test their features perfectly without QA assistance, QA is needed to reflect whether the <u>experience</u> is good. Are the response times reasonable? Is it <u>easy</u> (not only possible) to register? Is the newsfeed <u>interesting</u> (instead of merely working)?</span></p> <p class="x_MsoListParagraph" style="margin-left:1.25in; text-indent:-.25in"><span style="Courier New"; font-family:";color:#1F497D;" ><span style="">o<span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">QA is still the team to ask “what is the quality of the our <span style="font-weight: bold;">offering</span>?”</span></p> <ul><li><span style=" ;font-family:Symbol;color:#1F497D;" ><span style=""><span style="font:7.0pt "Times New Roman""> </span></span></span><span dir="LTR"></span><span style="color:#1F497D;">I believe that QFO can also take ownership of automating the tests, as we see in some cases. Developers will code the feature and QFOs will code the tests and hand them to the developers. </span></li></ul>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com3tag:blogger.com,1999:blog-1139266731001119488.post-6799867918482471502010-05-10T13:13:00.000-07:002015-02-04T00:19:32.483-08:00Prioritizing and automating tests<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
After choosing an automation platform and methodology, one of the greatest challenges QA/automation teams face is deciding what to automate and defining test priorities for automation. In <a href="http://www.delver.com/in?invite=friends-and-family">Delver</a> initially we asked <a href="http://omriqa.blogspot.com/2009/07/qa-feature-owner-when-testers-become.html">QFOs</a> for lists of Test Cases (TCs) to automate. The QFOs were supposed to compile lists, send them to Automation Lead who would review the TCs and allocate them to Automation Engineers (AE) to automate. In order to monitor this process, I used to measure the amount of automated tests Vs total TCs in the system. The amazing thing about this procedure was that it managed to frustrate everyone equally:<br /><ul><li>The QFOs kept feeling that hunting TCs in the system for the automation team was a time consuming burden. It was reoccurring task and often the AEs rejected the selected TCs for low ROI in automation.</li><li>Secondly, QFOs discovered that the regression tests that were needed for each version were not the ones automated. This was due to the fact that when the automation team were assigned TC to automate, the assigned cases were not correlated to any specific version, just arbitrarily selected from the TC pool.</li><li>The automation engineers did not manage to increase coverage rates. When comparing the automated cases to the TC pool, despite all the efforts, the impact was slim, and it was rarely over 35% coverage.</li><li>We also found out that of all the team in Delver R&D, the QA automation team were the only team who did not feel part of the release cycle. Their output was not correlated to any specific version or content and they slowly became "those dudes" who came in the morning, wrote a bunch of "automation things", and went home in the evening.</li><li>I became frustrated myself, seeing my talented automation engineers slowly drifting apart from the rest of the QA team, doing "their thing" without being able to relate to what everyone else was doing. Their feeling of belonging was waning slowly.</li></ul>So we changed everything. Instead of going after (and annoying the hell out of) the QFOs and demanding lists of TCs, we decided to automate the needed regression tests based on the known content of the upcoming version. In Delver we have a new version once in two weeks, so the AEs typically had two weeks to automate the needed TCs for the upcoming version. The list of tests for each version is compiled in advance anyway, so the only thing needed was to go to the list (we call them Test Runs, a list of Ad hoc TCs, compiled specifically for each version based on the contents of that version) take the most important TCs and start automating them, moving to TCs of lesser priority when to higher priority TCs were done. This left us in a more favorable position:<br /><ul><li>We no longer need the QFOs for setting automation priorities. We know what to automate: the tests we're about to run in two weeks.</li><li>QFOs immediately felt the relief. Since we started on automating the next version, when testing started, they had very little regression tests to run manually.</li><li>Automation team is now measured differently. Instead of measuring automated TCs Vs the whole pool of all available tests, they are now measured by the percentage of automated tests in each version. The numbers are significantly higher, reaching ~70% automation of regression tests.</li><li>Automation Engineers now feel they are part of the group because their schedules are now directly synchronized to everyone else's and their work affects the team within a week or two, not in some obscure future.</li></ul>Managing a team of developers in the QA group is challenging. The automation engineers' work is dramatically different from classic testing work and left unchecked, there is a risk they'll start drifting apart from the rest of the group. By finding a method in which the automation efforts directly affect our day by day testing efforts, we manged to bring them back to the team and dramatically increase their effectiveness on our daily work.Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com6tag:blogger.com,1999:blog-1139266731001119488.post-41453654346619572472010-05-07T05:14:00.000-07:002015-02-04T00:19:44.801-08:00Automatic tests in Delver: past and present<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
Our automation efforts started in May/June 2009 and have changed and evolved rapidly to their current form. In this post I'll review the process, what are we currently doing and where do we want to go.<br /><br />We started by programming Greybox testing. After reading a product's spec, we have developed code which tested each method from QA point of view, going deeper and broader than what developers tested in their unit and integration tests. We were quite happy for a while: we found bugs early, the bugs were fixed before the final version reached QA and we did not need to wait for the final version to start testing. Cool, huh?<br /><br />Not really. On July we had 3 testers writing code and one manual tester. However, researching the number of bugs found via automated testing versus bugs found manually it quickly became apparent that ~80% of the bugs were detected by one manual tester after the version reached QA while the remaining 20% was detected by three testers writing automated tests. What was the problem? Apparently many of the bugs were a result of missing components (the button wasn't there), broken components (the button <span style="font-weight: bold;">was</span> there but it didn't work) and other UI shenanigans. Yes, the backend was working quite efficiently, but the application was buggy as hell. If hell has bugs, that is.<br /><br />Another problem we found with Greybox testing is that we have realized we were writing tests developers were supposed to write. The developers liked it, of course, but we felt we are not doing our job.<br /><br />Next step was to switch to Automatic UI tests, or AUI. We have reviewed some solutions and finally settled on <a href="http://seleniumhq.org/">Selenium</a>, which we still use today and will probably continue using in the foreseeable future.<br /><br />We use AUI tests as an extension of our QA Feature Owners (QFOs, for short). AUI tests are tests written by the QFOs and are automated by our Automation Engineers (AEs) based on priority (for the QFO) and complexity of automation (for the AEs). Tests which are both of high priority and low complexity are automated. It is important to note that after tests are automated their execution is the responsibility of the QFO. QFOs are the driving force of Delver's QA team and AUI tests are tools at their disposal.<br /><br />One of the greatest challenges we faced when starting automate UI tests is switching manual tests to tests which are easy for our AEs to automate. In a way, manual tests are the complete opposite of automated tests:<br /><ul><li>It is effective to write long manual tests which cover several aspects/features of the application in one flow. If this approach is used in AUI tests, when a test fails, you have no clear indication of what exactly failed. For example if you mean to test feature A but you go through feature B and feature C on your way to A, if the AUI test fails, how can you be sure whether the test failed at A, B or C?</li><li>In manual tests there's no problem with copying and pasting blocks of instruction texts ("Login, , Search, click on button A") and modifying them per test ("Login, , Search, click on button B"). This poses a major headache for AEs when they struggle to pinpoint minuscule differences between tests (Button A in the first example, Button B in the second).<br /></li></ul>After understanding the problems AEs face, we have changed the way tests are written. QFOs write tests which are short, specific and very focused. For the sake of automation, we encourage writing multiple short tests over fewer, longer ones. This change wasn't easy for the QFOs, but they adapted rapidly. The alternative, I explained to them, was manually running their tests every two weeks.<br /><br />After the test is written by the QFO and we decide to automate it, QFO and AE meet and we make sure the AE understands what the tests does and does it test. The fact that all of the AEs are required to run some manual tests in each version also helps in that regard: the application is not "the QFO's problem" but a living entity they interact with on regular basis.<br /><br />After an AE is done writing the test, she is required to hold two reviews. A Code Review with a fellow AE, in which they run over the effectiveness of the code written in the same manner every code writing member of Delver R&D is required to hold Code Review. The second review is Test Review in which the QFO verifies that the code actually tests what it is supposed to. In the past, Test Reviews were painful to us because large gaps between QFO's intention and AE's execution were revealed in them but as QFOs began writing more automation oriented tests and AEs becoming more familiar with the application and testing focus, the reviews are shorter and easier now.<br /><br />Here is the procedure we use now for QFO/AE interaction:<br /><ul><li>QFO writes a test.<br /></li><li>If we decide to automate the test, QFO and AE meet in order to briefly go over the test and make sure the AE understands it completely. This meeting is not mandatory.</li><li>AE writes the test. This typically takes one day.</li><li>AE reviews the code with another AE.</li><li>QFO reviews the final product.<br /></li></ul>In order to be where we are today, QFO's needed to adjust the way they write tests to a way which is not optimal for manual tests and AEs were required to make sure their code reflects QFO's tests intention. We have come a long way in less than a year and I am currently happy with the results. If the QFOs are the heart of the QA team, automation are their cybernetic legs and arms allowing them to move and test faster.<br /><br /><span style="font-size:100%;">I just wish I could find a metaphor for QFOs shooting laser out of their eyes, that would have been so cool.</span>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com2tag:blogger.com,1999:blog-1139266731001119488.post-8724710194990144832009-09-08T01:41:00.000-07:002015-02-04T00:19:55.200-08:00QA Feature owner - when testers become project managers II<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
<span style="font-style: italic;"></span>We have been using QA Feature Owner (QFO) methodology for two months now so I guess it's time for me to review where this is going. My vision is very clear to me:<br /><ul><li>I (QA manager/Team Leader) don't want to decide what will be tested in each build</li></ul><ul><li>I want to increase testers' commitment and familiarity with "their" features</li><li>I want each tester to be skilled in all three testing methodologies we employ (Greybox/Integration tests, UI automation tests and manual tests) as well as with core QA methodologies and skills such as reading a specification document, analyzing it, deriving risks etc'</li></ul>If that wasn't clear enough, I'll now emphasize what team I <span style="font-weight: bold;">don't</span> want:<br /><ul><li>I don't want a team in which I have to decide which test will run in each new build. Too many test cases and too many changes in each build for me to do this effectively.</li><li>I don't want a team of domain experts who only test their area of expertise. I don't want the UI automation expert doing nothing but recording scripts and I don't want the person who is used to testing manually treat automation like black magic.</li><li>I don't want testers who test features when they assigned to them and forget them after the testing is done. I don't want tester who move from feature to feature.</li><li>I also don't want tester who got job security because they are the only ones who can "do this thing". This is not exactly QFO related so I'll deal with the versatility issue in another post.</li></ul><br />Since we started using the QFO methodology, we are testing differently. When we get a new build I sit with each QFO, asking her what she wants to test and why. Why is <span style="font-weight: bold;">this </span>test left out? Why do we run <span style="font-weight: bold;">that </span>test again? While I'm there to supervise, each QFO is responsible for creating "test runs" (that's how a collection of tests is called in <a href="http://www.mozilla.org/projects/testopia/">Testopia</a>, our testing suite) for new and existing features. QFOs run some of the tests themselves, other tests they delegate to other team members. That's OK, that's how the system works.<br /><br />This system is obviously not suited for every testing team. Team members need to be able to (and want to!) take responsibility over their features. Team members need to delegate tasks to other team members and perform testing tasks which aren't "theirs" so good communication skills are also required. It is geared towards responsible testers who want to be involved with the product and not just perform tests assigned to them by some authority figure.<span style="font-style: italic;"><br /></span>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com0tag:blogger.com,1999:blog-1139266731001119488.post-548028529333167042009-07-27T03:18:00.000-07:002015-02-04T00:20:18.891-08:00QA Feature owner - when testers become project managers<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
In our R&D organization, new features are usually led by developers. While managing the new feature, the leading developer is assigned the title Feature Owner for that feature. She delegates some work to other developers while performing some coding tasks personally. The larger the feature, more time is spent on managing and coordinating efforts and in smaller ones the FO spends more time coding. The interesting issue (for me) is the interaction with the testing team.<br /><br />Most features are multi tiered: there's a backend layer, some application layer and a UI. Tests for each layer are different, of course: Greybox testing for backend layer, SQL scripts for database related tests, UI Automation and manual tests for the UI, etc'. The problem is, most testers don't feel equally comfortable with each testing methodology so what tester do I sign for each feature? If I assign a manual tester to a "mostly UI" feature does it mean we want test the backend? My coder can't handle the UI automation suite, maybe I'll just skip automatic UI test?<br /><br />The other alternative is assigning several testers for each feature, based on their skills. This ensures that each tester is assigned tasks she's good at but it also poses other problems:<br /><ul><li>The overhead of testers learning new features is now multiplied by number of testers involved in testing the feature</li><li>Dispatching work between the testers and writing a test plan can not be done by the development FO, as she does not have the testing know how. Also, who is to know all the bugs related to the feature? Each assigned tester? Too messy and getting a quality overview is hard.<br /></li><li>Why not assign the QA team leader for dispatching tasks and writing a test plan? Because most of the time she will not do the actual testing on the feature, so we have a paradox: the QA person who studied the feature and designed the tests does not test it and the people who test it did not study it thoroughly (overhead, remember?)</li></ul>The solution we are practicing now in my testing team is assigning a QA Feature Owner (QFO) to the development Feature Owner (FO). The assigned QFO is usually the person who's skills are the most relevant to the feature, but she needn't posses all the required testing skills: the QFO will assign tasks to other testers in the team in areas in which she lacks or simply does not have time to run or design tests.<br /><br />This is how it works: upon reading the spec, the QFO designs a test plan, a document detailing the general testing strategy. In this document risks are identified and then tests are written as a sketch: a few words describing the general purpose of each test. We find it convenient to start by designing Greybox (GB) tests, then Automatic UI tests (AUI) and finally manual tests.<br /><br />It doesn't matter if the QFO is a programmer or not when designing GB tests. Using <a href="http://en.wikipedia.org/wiki/Data-driven_testing">Data Driven Testing</a> methodologies most GB test cases are list of methods and what they are supposed to do and attached CSV files which are no more than input/output files based on the original spec. AUI tests are simply stories in English ("go here, click there") which can also be augmented beforehand using CSV files. Test cases are later reviewed by other team members to validate their testing logic and enrich the tests in areas in which the QFO is weaker. For example, a manual tester can review the manual tests written by a QFO who has strong programming background and suggest additional tests. Test cases will later be examined by the FO and QFO together, in order to ensure full coverage.<br /><br />Once all test cases are approved by the QFO and feature and test coding begins, the QFO starts following the testing project. The QFO is responsible for ensuring that tests are run but not necessarily by the QFO personally: testing tasks can be assigned to other testers as long as the QFO remains the testing team's focal point for the feature. The QFO does not need to be the one reporting all defects but must know all defects opened which are relevant to her feature. The QFO is <span style="font-weight: bold;">responsible</span> for the testing of the feature but the actual tests may be performed by others.<br /><br />We have started implementing this methodology only a few weeks ago and it remains to be seen whether we benefit from it on the long run. I don't take the fact that we're asking hands-on testers to become project managers for granted and believe some will adapt to it better than others. Others will resent the time spent in meetings and reporting and will feel more at home getting their hands dirty with testing. I have no intention of forcing any tester to become a QFO, I consider it a perk, not a requirement. I do, however, believe that tester assuming the role of QFO will enjoy the challenge of managing a testing project, interacting with more team members from the R&D than they are used to and experience new testing methodologies. I believe a good QFO is a potential testing leader.Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com4tag:blogger.com,1999:blog-1139266731001119488.post-78899844688573059762009-02-05T01:53:00.000-08:002015-02-04T00:20:36.117-08:00QAing in a startup<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
After 8 years in big companies like Mercury (R.I.P) and Symantec I have started working for Delver, a startup company, in late 2007. After 15 months of starting and running the QA team I am wiser about QAing in a startup company.<br /><br /><ol><li>Testing is Testing is Testing. Good QA is about methodology: be methodic in your risk analysis and in your tests. This principal doesn't correlate to the size of the company. In Delver we won't write a test case to a feature if it's not documented. I don't settle for oral briefing or email overview. No document, no test case. Easy, really. On the same principal, no testing without a test case, even for small features.</li><li>Speed over quality. In a startup company the emphasis is on fast development with speed being first priority, quality second. This perspective dictates more investment in speeding up the testing process. We used the following methods:<br /></li>-Automation for testing relatively stable components<br />-Testing new components outside the main build if the build is not ready<br />-Issuing "non release" build for focusing on delicate features<br /><li>QA freedom/No safety net. Another aspect of speed oriented, developer dominated organization is the freedom for QA. Since the focus is usually on the developers the QA team in Delver enjoys more professional latitude than what I have experienced on larger organizations. The flip side is the lack of testing peers and no organizational testing history. There is no safety net for me in Delver and when I make a methodical mistake usually there will be no one to correct me. This is an exhilarating feeling, but it's also a scary one. Like all good thrills, I guess.</li><li>Smaller budgets. Big, commercial testing software is expensive. Startups don't like "expensive". Startups like "cheap" or even better: "free". On one hand this seems like a major limitation (oh, who am I fooling. It is a major limitation.) but on the other hand it's also a challenge and a cool one. Finding a 300$ alternative for a 80K$ load testing suite is a personal victory and shows me that I can overcome seemingly seemingly impassable hurdles (financial, for example). In the future I'll dedicate a post to cheap and free tools we use in Delver.<br /></li><li>No bureaucracy. Well, this one is obvious but it's still cool. When I need to ask something I can go to the guy and ask him. No email to another office, no phone calls to another continent. Go to the guy. Ask. End of story. If I want to hire a candidate I can give him a contract a day or two since the first interview. If I need a new license I simply go to the room of the guy who signs the checks and ask for the money. Simple. Fast.</li><li>More flexibility in reporting bugs. I don't open a bug for every problem we encounter. Since the company is a small one, when I'm not sure I can simply go to the room of the developer who is responsible for the code I found the problem in and verify whether it's a bug or not. This can be done in larger organizations as well but sometimes the relevant developer is not in the same building. Or town. Or continent.<br /></li></ol>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com1tag:blogger.com,1999:blog-1139266731001119488.post-37631746991993431502009-01-21T00:07:00.000-08:002015-02-04T00:20:50.371-08:00Half baked versions are good for you!<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
Often, we find ourselves in a situation in which a feature is ready but the version as a whole is not: It can take anywhere between a day or two to a week until all the developers have finished working on a build and all integration tests are concluded, but during this time some features can be tested. So we came up with a method for speeding up the testing process: testing incomplete builds.<br /><br />The idea is simple. When a feature is ready, we get an incomplete version. This version is installed on a separate, local, system and only the specific feature (or features) is tested. We don't test other components of the application and when bugs are found there we ignore them. This way, by focusing on specific, new, features, we are able to detect bugs before the release candidate version is released to the QA. Once we get the official version, we are more familiar with the new components and they are usually more robust as the most glaring bugs were purged earlier.<br /><br />Testing in this method helps us test more and more often, but one must bear in mind that testing this way can't replace thorough testing later when the full version is tested. Some bugs only rear their ugly heads in more complex environments and need to be detected separately.Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com0tag:blogger.com,1999:blog-1139266731001119488.post-45852547953513653672008-08-05T03:14:00.000-07:002015-02-04T00:21:00.568-08:00Testing in Chaos<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
At <a href="http://www.delver.com/">Delver</a> we work in <a href="http://en.wikipedia.org/wiki/Blitzkrieg">Blitzkrieg</a> mode: we work hard, develop fast and improve our asskicking search engine almost on a daily basis. Working like this is very exciting, I can sometime feel the imaginary wind in my face, but from QA perspective I have grown to realize that we have a problem testing efficiently in this development environment. Here's a list of the problems and questions I'm dealing with:<br /><ul><li>Sometimes the builds we get in QA should have never have left dev. The builds contain critical bugs which render the build untestable.</li><li>I'm not always sure how thoroughly we should test each build. Through testing of a new build is currently ~20 man hours for "core", previously tested, features. That's not including new features.<br /></li><li>I don't know when I'll get a new build. The longer we delay the build, the more time I have to test it as more features are added.</li><li>What amount of testing should be performed when configuration changes are done in Production?</li><li>If I test a build thoroughly on my lab, do I have to test it with the same amount of thoroughness when it is uploaded to Production? Where should I focus my testing efforts? Lab? production? Both?<br /></li></ul>With all these problems I sometimes feel I'm losing control of the process and that I react intuitively instead of in a planned manner. I want to test in the most efficient manner without changing the R&D flow. I want the best coverage for the minimal effort and I want to invest resources where needed, i.e. in new features and risky areas and reduce the amount of routine testing performed by my team. How am I going to tackle these problems:<br /><br /><ul><li>Develop a set of <span style="font-weight: bold;">Sanity tests</span> which cover all aspects of our application without going too deep into any of them. For example: Make sure Search feature works, but don't test special characters or long strings.</li><ul><li>These Sanity tests should be performed by developers before handing out new build to QA.</li><li>These Sanity tests should be performed by Operations team when they change a hardware configuration and want to make sure they didn't break anything.</li><li>We will automate these tests and instruct Operations and dev how to run them.</li></ul></ul><br /><ul><li>We have a rather impressive set of <span style="font-weight: bold;">Regression tests</span>, most of them automatic. I'll divide them into two levels:</li><ul><li>Level I (Core): The Core of Regression tests. Deeper than Sanity tests, faster than Full Regression. All automatic tests are Core but not all Core are automatic. When do we test Core tests?<br /></li><ul><li>When a build is not Production candidate, or<br /></li><li>When only minor changes were made since last Build</li></ul><li>Level II (Full): All Regression tests. When do we test Full Regression</li><ul><li>When a build is a Production Candidate, and</li><li>Major changes/features are implemented in this build</li></ul></ul></ul><br />To summarize, here is the procedure we'll use regarding new builds and configuration changes in delver:<br /><ul><li>Every new build should pass a Sanity test by dev before it is ready for QA.</li><li>When Operation changes configuration or install minor fixes which did not pass QA, they should also run Sanity tests.</li><li>QA will run Core tests on new builds unless the new build is both Production candidate and major changes were performed since last build. In that case, Full Regression tests will be performed.<br /></li><li>New features will tested thoroughly on the build they become available. In later builds they will be tested either as part of Core or Full Regression tests.</li></ul><br />I'm still not sure about how much resources should I invest in testing in Production versus Lab, but that's a matter for another post. Sometime in 2009, probably.Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com0tag:blogger.com,1999:blog-1139266731001119488.post-54542790146480933302008-03-16T08:04:00.000-07:002015-02-04T00:21:11.677-08:00Performance testing mistakes (I've made)<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
I've been testing for Performance for eight years now, more or less. When I started testing, I didn't even know I'm supposed to test for "Performance", I was simply the- dude-who-knew-VUGen. Neither me, nor my manager cared much about performance testing methodologies: I'd simply record a script, run it with multiple users and report my findings. This is a very bad attitude to run performance tests, as I have found out in several heated QA-dev meetings. The worst part was seeing my credibility deteriorate, and my hard earned findings doubted. Over the time I managed to pinpoint my mistakes and win my credibility back, but the process was long and painful. In this post I'll discuss some of the biggest performance methodology mistakes I have committed over the years.<br /><br /><br />1. <span style="font-style: italic;">Configuration problems</span>. This one was the hardest on me. You run a test, you find what you think is valuable performance issue, you call some developers and have them investigate your system, only to find out that you ran out of disk space. Or the debug level was to high. Or the memory wasn't properly configured.<br /><br /><span style="font-weight: bold;">Always</span> verify your system before a test. Read all the Read Me's, even the ones no other QA tester reads. Same goes for Best Practices. In one of the companies I worked for, we found a DB configuration document no one even knew existed, and we configured our system properly. Prepare a checklist a verify it before each test. Misconfiguration will bite you. Hard.<br /><br /><br />2. <span style="font-style: italic;">Mixing conclusions with findings</span>. I love monitors. I disperse them generously across the applications and servers and monitor all relevant process (and some which aren't). In the end of the day, I sometimes have 100-200 monitors per server. While this is not a problem in itself, it became one when I didn't know what to report.<br /><br />When asked to present my conclusions, I used to report anything which held a trend: Working Sets increasing, Context Switch peaks, Threads amount changing. The graphs looked impressive and I felt like I have a lot of "meat" in my reports. Problem started when people started asking: "well, what does it mean?" , "isn't it normal?" or even "is it good or bad?"<br /><br />In many cases, I had no idea. I figured that other R&D personnel knew what those stats meant even when I didn't. Well, they didn't either.<br /><br />Many trends are cyclic by nature (resource consumption is increasing and then decreasing), some reach a flat plateau after increasing and some rise and fall like saw teeth. In most cases, Monitors of hardware usage only give you hints of the problem and rarely the problem proper. There are some exceptions to this rule: high CPU consumption over a prolonged period of time is bad in most cases, for example.<br /><br />The findings are important, as they help know your application better and know it baseline. If you're already familiar with specific monitors (I/O of process x, for example) following its trend can help you detect when an application is misbehaving, but in most cases monitors results are best left out of your conclusions section and kept in the findings baseline. Report only things which are definitely bad and you're sure about.<br /><br /><br />3. <span style="font-style: italic;">Fighting your previous war</span>. There's a saying about the army that it's always preparing for the previous war instead of the next one. While it's always fun to make jokes about the army, most people handle the current project similar to the way they handled the previous one. I once had my engineers spend a week trying to locate memory leaks which weren't there, only because the previous product I tested suffered from memory leaks. In another time, it took me some time to adjust to the fact that the load on the system I tested is not caused by user activity, but by amount of data flowing into the system. Bye-bye 100 concurrent users load test.<br /><br />Performance problems on different systems are caused by different factors affect different components. Study the system you test, learn what hurts it and how does it express its pain. Design your tests after you are familiar with the system and try to verify with yourself whether you're testing what you're testing because you're used to it or because it's relevant to the current product. Fight the current war, not the previous one. Or you can join the army if you like the uniform, though.<br /><br />A subsection of this issue would be to test what's easy for you test. Nice try, but no. Don't.<br /><br />4. <span style="font-style: italic;">Convoluted reports (no one reads)</span>. I'm not intimidated by putting together a 200 pages summary document, I actually take a perverse pleasure in it. It had taken me some time to understand no one really reads them if you simply throw the reports in their faces (or in their inbox, for that matter). Having a 10 MB, 200 pages monster uncoil before your eyes is a scary sight when you're unprepared and the natural reaction of most people I sent the report to was to close it and "read it later".<br /><br />So I changed the format of my reporting and now my reports get the attention they deserve (most of the time, anyway). R&D managers are like ADHD children: they have short attention span and they like bright and shiny things, so in order to get their attention I compose a mail they can easily digest. The secret is to put nice <span style="color: rgb(255, 0, 0);">c<span style="color: rgb(153, 0, 0);">o</span>l<span style="color: rgb(51, 204, 0);">o</span>r<span style="color: rgb(51, 255, 255);">f</span>ul</span> graphs and <span style="font-size:130%;">simple explanations</span> summarizing your most important conclusions in the mail you send to the relevant people. No more than 2-3 issues. Put the rest of the report online somewhere and attach a link to it. Most people will ignore it, so you better make sure that executive summary count. <span style="color: rgb(255, 204, 51);">C<span style="color: rgb(102, 0, 0);">o</span><span style="color: rgb(0, 0, 153);">l</span><span style="color: rgb(204, 0, 0);">o</span><span style="color: rgb(51, 204, 0);">r</span><span style="color: rgb(102, 102, 102);">f</span><span style="color: rgb(51, 0, 0);">u</span><span style="color: rgb(255, 204, 255);">l</span></span> graphs, simple explanations and no more than 2-3 conclusions. Easy.Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com2tag:blogger.com,1999:blog-1139266731001119488.post-46587923990209105682008-03-10T00:45:00.000-07:002015-02-04T00:21:22.069-08:00What's QA?<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
<div dir="ltr" style="text-align: left;">I had a not-too-bright manager once, who told me that the job of the QA was to ensure the quality of the product. I told him that I can't do that: I can ensure the quality of the <a href="http://img217.imageshack.us/img217/5058/ifo2rl2.jpg">miniatures I paint</a>, the salad I cut or, to some extent, the manner of my daughters. I can't ensure the quality of code I don't write, it's that simple. Ensuring the quality of the software is not my job.<br /><br />I often see QA engineers who feel personally offended that software is released with (some) open bugs, or that their bugs are re-categorized as enhancements. I always tell them that being offended by the way our bugs are treated is not our job.<br /><br />In the organization I currently work, I encounter Dev personnel design and run tests. I was surprised and a little offended at first, but I soon realized there's no reason for that. Hell, even testing is not my job (though I'd better be good at doing it).<br /><br />So what's my job, actually? <span style="font-size:10%;">It's not <a href="http://www.youtube.com/watch?v=QenJDBmxi1E">eating these donuts</a>.</span><br /><br />In Delver, we have a white board in the meeting room. My job is to go to that board once in a few weeks and write the <a href="http://img256.imageshack.us/img256/9628/photopj2.jpg">product's status</a>. I give a <span style="color: rgb(51, 204, 0);">smiley</span> to a component which is ready for production, <span style="color: rgb(255, 204, 0);">so-so</span> face to a component which still has some bugs before it's ready and a <span style="color: rgb(255, 0, 0);">frowning</span> face to a component with critical issues. That's my job as a QA manager. Go to the board and paint some faces with a dry erase marker.<br /><br />Because the role of QA is not to ensure the quality of the software, not to fight for bugs and not even to test the product (yes, I'm being a wise ass here). The role of QA in organization is to be able to <span style="font-style: italic;">reflect the quality of the software</span>. In order to do that we have to test or help the dev personnel test. We must be efficient, resourceful and clever in our tests. We must know the requirements and understand how the users use the software. But in the end of the day, our job is to reflect the product's status.<br /><br />Are there critical bugs you feel are ignored? Publish them. Make sure everyone who should know about them, knows. Should you care about known defects the software is released with? of course, you're part of the team. Is it your job to stop the version? No. Your job is to make sure that the people who make the decision about shipping it know everything they should know.<br /><br /><br /></div>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com2tag:blogger.com,1999:blog-1139266731001119488.post-81136253457945491672008-03-09T10:28:00.000-07:002015-02-04T00:21:30.256-08:00Here we go<script type='text/javascript'>
DY = {scsec : 8765695 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};
</script>
<!-- BEGIN DY SCRIPT TAG FOR SECTION 8765695 - DO NOT MODIFY -->
<div id='DYSCR'></div>
<script type='text/javascript'>
(function(){var d=document,e='createElement',a='appendChild',g='getElementsByTagName',m='getElementById',i=d[e]('iframe'),l='localStorage';try{window[l]}catch(ex){l=''};
i.id=i.name='DY-iframe'; i.style.display='none'; i.width=i.height='1px';d[m]('DYSCR')[a](i);
DY.x = function(w) { var d=w.document, s=d[e]('script');s.type='text/javascript'; s.async=true;
s.src=('https:'==d.location.protocol?'http://st.dynamicyield.com'.replace('http:','https:') : 'http://st.dynamicyield.com')+'/ast?sec='+DY.scsec+'&id='+(w[l]&&w[l].getItem('_dyid')||'');
d[g]('head')[0][a](s);}; var c = i.contentWindow.document;
c.open().write('<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"><body onload="parent.DY.x(window)" style="margin:0"></'+'body></html>');
c.close();})();
</script>
<!-- END TAG -->
<div dir="ltr" style="text-align: left;">I test software for roughly eight years. I learned a lot and I still do. I have started this blog in order for me to help others, talk about the stuff I like and rant about the stuff I dislike. I believe I'll focus on Performance testing which I love and QA in Startups, which is the challenge I'm currently facing. I intend to keep this blog mostly professional, but I can't help it if I mention my family. My daughters are simply <a href="http://picasaweb.google.com/olapidot/2008/photo#5170159025646259378">too cute</a>, I apologize in advance.<br /></div>Omri Lapidothttp://www.blogger.com/profile/17133432243214271455noreply@blogger.com0