Selecting Qualified Investigators

The focus of this post will be selecting adequately performing US clinical trial sites for Investigational New Drug studies.  Depending on the nature and size of your clinical trial, you may be targeting private practices, specialists, academic centers, purpose-built dedicated clinical research facilities, or a combination of any of these to find your principal investigator(s) (PI).  Certainly, there is great cost associated with opening and maintaining a study site so you are well advised to be thorough and thoughtful when evaluating the potential investigators.  As monitors, we may provide input on the clinical operations of a study and assist or manage projections of how many and what types of PIs will be selected and this article will discuss some of the important considerations for qualifying and selecting study sites.

As qualified CRAs, we use our instincts and training to predict and choose the most qualified and motivated PIs.

Inevitably, some study centers will perform better than others and deliver the ideal study candidate while strictly adhering to the protocol.  Careful site selection is absolutely critical to limit the time and resources that will be required during study conduct to manage poor performing sites; I refer to my non-enrolling or poor performing study centers as “duds”. Duds require additional monitoring oversight, coaching, and avoidable cost for continued participation, therefore, these sites are often closed before enrollment or conduct is complete.  Closing a non-performing site is an attractive option to suspend the siphoning of time and resources, but forethought and a well-coordinated site selection exercise can help ensure dud sites are never opened in the first place.

Consider the therapeutic area of study and recruit PIs with relevant experience, the right study tools and equipment, access to the target subject population, interest/motivation to recruit, and availability to participate.  

You needn’t only consider PIs with prior clinical trial experience, but you may gain some considerable efficiencies by targeting PIs who have performed well in prior studies and audits.  Some PIs pay to be listed in clinical trial directories, you may be able to obtain names of doctors to approach through online physician directories, reviewing PIs you or your colleagues have worked with in the past, internal company databases, conference attendee lists, patient advocacy groups, journal article author searches, pharmaceutical sales personnel, medical association websites, telephone directories, the internet, trade journals, other networking contacts, etc.

The FDA maintains lists of disqualified investigators and those that are prohibited or restricted to conduct clinical trials.  You can also review any previous warning letters or license restrictions using publically available databases (Click on the hyperlinked text and see additional links below for further information).  Your investigators will need to adhere to all ICH/GCP guidelines and relevant CFR sections including 21 CFR 312.53.  Check your potential PIs against these lists and also gather anecdotal information regarding the investigator from colleagues and the internet. Read physician review websites like yelp.com or clinician directories.  Determine if there have been previous inspections at the study center.  Some doctors have their own websites and you can also search news databases to find information about a potential PI you are considering.

http://www.fda.gov/ICECI/EnforcementActions/FDADebarmentList/default.htm
http://www.fda.gov/ICECI/EnforcementActions/DisqualifiedRestrictedAssuranceList/default.htm
http://www.accessdata.fda.gov/scripts/cder/cliil/

http://silk.nih.gov/public/cbz1bje.@www.orilist.html

Relevant official FDA guidelines are available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM187772.pdf  and Section C of http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm070291.pdf

Here is a checklist from one of my trusted colleagues:
http://www.ehow.com/how_2352851_choose-good-clinical-research-investigators.html

Finally, you can contact the physician and his staff to inform them of the trial, determine their interest, and request that they fill out questionnaires or provide a copy of their CV or a summary of their experience.  If you do develop site selection questionnaires you can ask them for their input on the trial design, to describe their facility, to quantify their qualifications/experience/availability/resources/staff, to identify if they have the proper equipment, to comment on their previous clinical trial or audit experience, to give details on their IRB process, to project potential recruitment and access to the target subject population (double check that there aren’t competing trials at the facility or within the same geography www.clinicaltrials.gov), to detail translation and advertising needs, and to inform you of any anticipated barriers to recruitment.  Much of this work can be done via telephone/fax/email but you may want to schedule a time to visit their office and evaluate whether they are a fit for your trial. For more information read my post on Pre-Study Visits.

Beyond the questionnaire, you can use the site selection process to gauge the communication style, organization, and responsiveness of the site staff.  

Beyond the questionnaire, you can use the site selection process to gauge the communication style, organization, and responsiveness of the site staff.  Ensure there are no conflicts of interest, lack of objectivity, or other reasons to avoid a specific PI.  You will be able to determine the PIs experience conducting other sponsored trials, develop a feel for the turnover at the site, and sniff out any additional red flags that may steer you away from choosing that site or anticipate issues that might creep up during conduct.  Be tactful, gracious, and considerate of their time.  The PI will be evaluating whether they want to work with you as well.  It is very much a two-way street since you may be working closely together for a considerable amount of time if they do join the trial.

Good luck selecting qualified investigators. Definitely take the time before the trial starts to be thorough and deliberate with the site selection process.  There is always pressure to get sites up and running as quickly as possible, but you must balance that need with the requirement to spend adequate time selecting qualified investigators. Hopefully spending enough time choosing your PIs will ensure your trials will be easier to enroll, your studies will be conducted with less protocol deviations, and your study team will be more likely to avoid unnecessarily costly “dud” sites and achieve all of your study objectives.

You may also like...from The Lead CRA archives:

• Routine Monitoring Visits March 2008
• Investigative Site Recruitment Plan Sep 2010
• Motivating Study Coordinators Feb 2011

CRA Orientation and Training

I have been a Clinical Research Associate (CRA) for three years, but I recently accepted a new CRA position and thought it would be helpful for me to explain the on-boarding process. Whether you are working for a CRO, pharma/biotech, or as a consultant, there is always new hire orientation and training to complete. The CFR and ICH/GCP require drug companies to ensure that the monitors like me are qualified by training and experience. In order to demonstrate our compliance we must document that training. Our expertise is also documented in a signed copy of a CV.  Finally, we typically sign and date a copy of our company furnished Job Description to be saved in the training or employee files that serves as an acknowledgment that we understand what is expected of us in our role as a CRA.

I blogged a few years back about my experience in my last New Hire Orientation but I will elaborate today on what types of Standard Operating Procedures (SOPs) and training courses a CRA might reasonably expect to complete at any company. So the important thing to note is that you repeat some form of this training at any new company whether you have one year experience, five years experience, or twenty years experience.  In fact, even within a company you may find that you repeat these trainings annually or periodically - this protects you and the company because you have a documentation trail demonstrating your qualifications to monitor clinical trials.

Everyone signs off on the required training before they step foot into a clinical site. This protects you as well so be wary of any company that wants you to go out and monitor without proper training and qualification documentation in place - it really can be a risk for you.

Companies typically offer new CRAs administrative training, regulatory/procedural training, and project specific training.  Trainings can be administered via a traditional classroom setting, via webinar/teleconference, over the internet using recorded playback sessions, in person, or on-the-job.  Obviously there are pros and cons to the different formats (opportunity to clarify and ask questions, assess understanding, consistency of training delivery, level of interactivity and engagement, etc. but that is a bit off-topic for this post).  Irregardless of the format, at the end of each training some documentation is typically generated and signed (may even be signed electronically) to detail who led the training, who attended, the topics covered, and the date of the training.  You may be in training for a few days or even a few months.  The formality of a new CRA training program is very company specific but I am going to attempt to generalize the three categories of required training that I have experienced.  This is not exhaustive and certainly these may not all apply to you:

CRA Administrative Training
General: How to Order Office Supplies, Completing Timesheets, Expense Reports, Travel Policy

Information Technology (IT): Accessing the Computer, Intranet and Email Policy, Company Specific Websites or Applications, Using Microsoft Outlook/Lotus Notes, Setting up Voicemail and Email Signatures, Electronic Data Capture (EDC) System

Human Resources (HR): Company Overview, Employee Benefits, Performance Review and Employee Development System, Ethics and Compliance, Insider Training, Conflicts of Interest, Privacy Policy, Dispute Resolution, Harassment/Sensitivity Training, Ergonomics, Workplace Safety

Regulatory Procedural Training
Regulatory: 21 CFR 312 & 812; 21 CFR Parts 11, 50, 54 & 56; ICH/GCP; Good Documentation Practices; HIPAA/Confidentiality

SOPs: Informed Consent Process, Investigator Site Selection, Pre-Study Selection Visit (PSV), Site Initiation Visit (SIV), Routine/Interim Monitoring Visit (MV), Close Out Visit (COV), Source Document Verification, Record Keeping and Retention, Electronic Signatures, Fraud and Misconduct, Protocol Deviations, Audits, Inspections by Regulatory Authorities

Project Specific Training (may overlap SOP and general training)
Writing a Monitoring Report
Monitoring Plan
Trial Master File Maintenance
Completing Site Contact Records
Case Report Form (CRF) Completion Instructions
Integrated Voice Response System (IVRS)
Using Diaries, PDAs, or other Patient Reported Outcomes (PRO) Instruments
Protocol Training
IRB & Regulatory Submissions
Study Budget
Investigational Product Handling and Accountability
Randomization and Unblinding
Study Supply Management
Serious Adverse Event (SAE) Reporting

I hope this overview was helpful. I am happy to expand on any of this - just comment or email to let me know if you have follow-up questions.

What to Expect at a CRA Interview

I’m posting to let you know what kinds of questions you might expect to be asked during a CRA interview. Practice these in the shower, in front of the mirror, or with a friend so you are prepared!


Typical Questions
Here are general questions that you will likely be asked with tips for answering shown in italics:

• Tell me about yourself? Keep it short and hand them a copy of your resume to look at while you give the penny-tour of your background and experience. Showcase yourself, “I live in San Francisco and I have 5+ years experience in this industry. I have been working as a CRA for over 18 months. I have had the opportunity to work in Phase I, II, and III trials for the following indications… I attended courses for CRA training through a local university and I am an active ACRP member.” Be prepared to explain why you are currently job seeking and any gaps in your employment history or a trail of many past job over a short period of time.
• What are your salary requirements? I’ve had success with this script, “I will consider the entire package including employer flexibility, 401K, a competitive benefit package, etc. However, my minimum salary requirement is $$$$. I would be pleased by a compensation package that includes benefits and incentive compensation to increase the value. From my research, it appears that similar positions in this industry are paying between $$$$ to $$$$.” Then be quiet!!!
• What percentage/amount of time are you willing to travel? Be honest. Travel is difficult and it is not in yours or the employer’s best interest for you to misrepresent yourself when this question is asked.

So here are the more role-specific questions:

• Have you had GCP training or formal CRA training through an employer or accredited course?
• Do you have experience with the following?
  o Regulatory document review
  o Selecting and recruiting qualified and interested investigators
  o Performing source document verification
  o Utilizing EDC, electronic diaries, or IVRS systems
  o Database close
  o Development of protocols or ICF?
• Do you have experience with the following therapeutic areas: Cardiovascular, CNS, Oncology, Diabetes, HIV, etc.? If you are interviewing with a CRO this is especially important because they will need to market you internally and to clients. There is no sense in lying here because there is no substitute for experience. Try to highlight experiences you have from other jobs that better prepare you for a monitoring role. For example: “Although my oncology experience at this point has been limited to clinical database and CRF design, that experience has afforded me the opportunity to read through no fewer than 10 to 15 different oncology protocols from a variety of sponsors and assist (at least on the data end) in these sorts of trials. I have had the opportunity to develop some very fundamental and transferable skills in XYZ indication this past year related to study start-up, maintenance, building/maintaining relationships with study sites, trial fiscal management, and facilitating cross-functional trial specific working teams to track and deliver important study milestones on time/budget.”
• What is the greatest number of [protocols, sites, patients] that you have been responsible for at any one time?
• Which phases have you worked in?
• How many [pre-study qualification visits, site initiation visits, site monitoring visits, site close-out visits] have you conducted in your career?

Undoubtedly, the last question you will be asked is, "Do you have any questions for me or about our company?" And the answer is, Yes, you do! I know you will be nervous, but ask good questions and listen to the answers to the questions and try to get feedback during the interview so you know how you are doing.  You may have been misunderstood during the interview or missed an opportunity to really sell yourself.  Asking great questions will give you one last chance to show your best or address any concerns the interviewer has before you part or even in a follow-up letter or email.  Here are some generic ones, but you have to ask something:

• Describe the three top challenges that I'll face in this job?
• What are the key metrics for measuring success in this position?
• If I were offered the opportunity to work with your organization, what would you like me to accomplish in my first 90 days?
• How would you describe the qualities of the most successful people at your company?
• How closely do my qualifications match the requirements for the open position?

Interview Styles
At this point, I want to mention that there are various interview styles. Being familiar with how (and why) questions might be asked will only help make you more prepared when it is time to come up with the answers. I am going to discuss a few interview styles below. The person conducting the interview may stick to just one style or have a mix of several – this is especially true when you are in a panel interview. Then it feels like you are in a firing range but just stay calm and try to engage the panel so you aren’t doing all the talking. After all, this is your opportunity to interview them, too, right? Anyway, here are a few interview styles you might encounter:


Behavioral – Everyone knows that past behavior is a very good indicator of future performance. This interview style will require you to answer in a manner that shows you are creative and quick on your feet. Sometimes the right answer is not as important as the delivery. Before you answer think hard about why they are asking this particular question and try to cite examples of past behaviors that are relevant to the job you are applying for and explain at the end how the skills and experience you are describing would be transferable in your new role. If they ask you about a past mistake or you describe something you did but would now do differently, say so and explain why. Nobody is perfect so showing that you learn from past experiences can only help you in the interview.

• Tell me about a time when you showed your ability to [adapt to a new situation, solve a problem, etc.].
• Tell me about a time when you demonstrated [initiative, integrity, excellent communication, etc.].

Situational/Scenario – If you have no idea what the right answer is, you can try stalling, asking the interviewer for help, or just qualifying your response. For example, “Honestly, I am thinking of a few possible answers for the question. On the one hand, I feel like this might be the right response, but on the other hand, I can see a scenario where this might be the right approach. Which are you looking for?” or “Well, I must admit that I haven’t ever been faced with this particular situation, but I imagine I would handle it as follows….” Here are some examples of situational questions:

• You’re at a site and discover an unreported SAE, what would you do?
• You are performing a routine monitoring visit and discover that the site has enrolled a subject but forgot or improperly consented them, how would you handle this situation?

Assertive – I personally find this style offensive. Any employer that utilizes these tactics is likely not one that I am very interested in working for. After all, if they are a jerk in the interview, what is it going to be like on a daily basis or when it is time to sign your expense reports, or partner to work on your performance goals, etc. In any case, an assertive interviewer will treat you really nastily or act very cold/aloof to see if you will get nervous or flustered. I find brevity is key in these types of interviews and if there are long periods of silence, just sit calmly and pretend like you are unaffected. Be careful not to be too defensive – it is usually a trap to see if you can keep your cool under pressure. They might say something catty like:

• It sounds like you don’t have a lot of experience with [insert something rude here], what makes you think you can do this job?
• I have a lot of other more qualified candidates, why should I hire you?

Structured – This interview is basically just like a test and you need to pass to make it to the next round. It is a typical style for CROs (where many different people perform new candidate interviews but then need to assess all applicants against the same standards) or for first-round interviewers. Essentially, the interviewer is looking for a specific answer, and once you say the magic word or list the 3 bullets they are looking for, they will move on to the next question. It is fine to ask how you are doing if you are feeling nervous along the way – they will probably find it endearing rather than count that against you (they probably suffered through the same type of interview when they were hired).

• What are the critical components of an ICF document? Know your regulations and GCP guidelines – All of them – this one is CFR 21 Part 50.
• What are the activities that take place during an [initiation visit, close out visit, etc.]?

Conclusion

Every person you interact with may be
asked to provide input on your interview
performance.  Remember to always respect
"gate-keepers" like reception/administrative
staff and the HR person who arranged
the interview.

Here are a few other random tidbits of general interview advice:

• Show up on time, be well-dressed/groomed, and be nice to the receptionist.
• Fill out the employment application in advance. When you ask for this you are demonstrating your enthusiasm for the position and you are showing how organized and methodical you are when approaching new situations. If there is no application to be completed, at least bring your government issued ID and a list of references and old employers in case this is needed.
• Avoid sounding cocky, smug, or coming off as entitled.
• Assess how the interview is going by using any of these targeted questions after you provide an answer: “Did I give you enough detail?”, “Was I clear on that?”, or “Would you like me to elaborate?”
• Never ever ever lie about your experience! Our industry is a small one and your reputation will precede you. It is a huge risk so just avoid it. Besides, as CRAs our primary objective is to ensure the safety of subjects exposed to Investigational Product. As an extreme example, if you lack the experience to do something and say that you actually can, you may be given a position that you are not qualified for and as a consequence, subject safety could be jeopardized.
• Never say anything nasty about a previous co-worker or employer – it only makes you look bad.
• Watch for and avoid illegal questions – It is absolutely inappropriate (and in some cases against the employment laws) for the interviewer to ask you questions about your age, race, religion, marital status, and whether or not you have kids. If you choose to volunteer this information in the course of an interview, fine. However, if you are queried about any of the items above just say politely, “I’m sorry, I don’t feel comfortable with that question but would be happy to address any others you have.” or re-direct the questioning in some other tactful way.
• At the end of the interview thank them for their time, ask what the next steps are in the process, and ask for the job! I know it sounds silly, but 90% of applicants won’t ask for the job. You don’t need to be shy. You researched this company for weeks, you got all dressed up and took an entire day (or several) to come in and interview. You don’t need to dance around why you are there, “When can I start?”

Now you know what kinds of interview questions to expect but this is really putting the cart before the horse because we haven’t even scratched the surface of how to execute a sound job search. I will be posting in the future with more information that will make you better prepared for your new CRA role including: ‘Choosing the Right Company as Your New Employer’; ‘Networking’; ‘Doing Your Homework Before an Interview’; ‘Following up After the Interview’; and ‘Tips For Excelling in your First 90 Days of Employment’. Let me know if there are other topics that are of interest and I will be sure to address them in other future posts.

You may also like...from The Lead CRA:

• How to Become a CRA January 2009
• You Might be a CRA if...December 2008
• What are your Weaknesses? June 2009
• Challenges to a New CRA May 2009

• Introduction to Monitoring December 2007
• Creating a CV November 2007
• New Hire Orientation November 2007

Staying Flexible and Creating my CV

Working at home isn't all bad but I
have to stay focused and shoo off
distractions like my little yorkie, George.

I have been kicked back at home in my bunny slippers pouring through the SOPs and online training courses for my new CRA gig. As of orientation, I was assigned to one study but now I have been picked up for 2 different ones (luckily in lieu of the first rather than in addition!). One of the first tasks every new CRA at a CRO must complete is entering your Curriculum Vitae (CV) into the corporate database. CV is fancy biotech jargon for résumé (Wikipedia tells me that CV is Latin meaning "course of life" and résumé is French meaning "summary"). Basically, you want to chronicle all of your past relevant work experience but more importantly, you must complete an inventory of all of your relevant skills and therapeutic experience. I had to indicate which Phases of clinical trials I have experience in, what types of studies I have worked on, and quantify everything...3 months? 6 months? 1 year? The other fun part of the CV was the summary section where you write a little paragraph about yourself in the third person.

BIOGRAPHY
Nadia BoBadia, BS is a CRA II responsible for monitoring clinical studies with various indications. Before joining the company, Ms. BoBadia was employed as an in-house CRA I at a specialty cardiovascular biotech company, where she managed site budgets/negotiation,forecasts/accruals, and processing/disputing invoices, along with monitoring 3 Phase I cardiovascular and pulmonary trials. She traveled internationally to support FDA audit inspection preparation visits and NDA triggered activities for 2 large Phase III studies. For two years Ms. BoBadia was employed at a CRO and an Electronic Data Capture (EDC) company performing data management functions including designing CRFs and clinical study databases for a variety of therapeutic areas. Prior to that she was employed at one of the top 3 pharmaceutical companies and a large biotech performing commercial product and field sales personnel support for pulmonary and CNS indications. She has additional experience as a project assistant on a Phase III Ophthalmology study. She joined the company in 2007 as a CRA II.

So this electronic CV is what the sponsors (pharmaceutical and biotech companies that have contracted with my service company to run part or all of their studies) use to decide whether or not you are appropriately trained to work on their studies. So even though I have a job, it is like I am constantly interviewing in order to be able to do my job (get billable hours/activities). I know in time my skills inventory will be so impressive that sponsors will be delighted to pick me up for their studies so that is definitely satisfying.

Half of the folks from my training class are already out traveling and monitoring but I have to finish my study specific training (reading the protocol, monitoring plan, sponsor SOPs, etc.) and receive my site assignments. Right now I am projected to start traveling the week of Dec 2nd so I am just enjoying the downtime while it lasts. I have been enrolling in benefits, setting up my home office, playing with my new printer/fax/scanner, ordering office supplies, and generally keeping myself busy.