Operator: Good ladies and gentlemen and welcome to the SY 2004 Safestart Promising Approachs for children exposed to violence audio teleconference. At this time, all participants are in a listen only mode. Later, we will conduct the question and answer session and instructions will follow at that time. If anyone should require assistance during the conference, please press "*" then "0" on your touch-tone telephone. As a reminder, this conference is being recorded. I would now like to introduce your host for today's conference Ms. Katherine Darke Schmitt Schmitt, please go ahead.

Katherine Darke Schmitt: Good morning, this is Katherine Darke Schmitt from OJJDP. We have first an introduction from my colleague Bill Schechter of JJ Safestart Program Staff.

Bill Schechter:: Good morning everyone. Thanks for participating. I would like to just give you an overview of what we are going to try to do in this next hour and half and you've already heard that -- that you are essentially on mute until you are given instructions on the question and answer portion. The first thing we are gong to do is spend a brief amount of time just talking about the agenda and the parameters for the call and so forth and then we are going to answer a couple of e-mailed questions that came in. And then open it up for about 15 minutes of questions and answers on -- on program announcement as a whole. After that, we are going to turn to the evaluation side of things and Katherine, who is the evaluation coordinator for this -- for this program announcement, will offer a few remarks about the evaluation data collection aspects of the -- of the program and then open it up for questions and answers about data collection and evaluation plans. And then she will transition to a presentation by two of the national evaluators for what we call phase one of the Safestart demonstration, those eleven sites that are currently operating right now. Those sites you may recall are listed in the program announcement and organizations and agencies from those jurisdictions are not eligible to apply on this what we call phase two promising approaches. I would like to let you know, who is going to be speaking this afternoon. Obviously, you have already heard from Katherine Darke Schmitt Schmitt, I am Bill Schechter and we also have Cecilia Duquela Fuentes, we are all three from the Office of Juvenile Justice and Delinquency Prevention. And then we have representatives from the two national evaluators of the current Safestart demonstration phase one. David Chavis from the Association For the Study and Development of Community and from Caliber Associates we have Janet Griffith, Lisa Lunghofer, Susan Chibnall and Kate Abbruzzese. So that's who you will be hearing talking and answering questions today. I think it would be helpful if everyone had in front of them either a hard-copy on the computers. The program announcement and the frequently asked questions. The program announcements about 53 pages long and the frequently asked questions which are in the process of being updated, I don't know if they are most recent updated there, are current through at least last week. Let see the -- there is also a very good source of information particularly about the evaluation site of things on a web site called capacitybuilding.net and the -- the URL for that is www.capacitybuilding.net. Bear in mind that these are materials relating to phase one, the eleven existing demonstration sites and not and they are somewhat different in their purpose and length. For instance they all had year and half planning period, the second phase does not envision, such a planning period. They are basically five to six year long programs and this is envisioned as to up to four years long. And they have in addition to delivering services and -- and providing other kinds of services to the community at large, systems reforms, policy analysis, policy change aspect. So there are some difference that you should bear in mind as you read the materials and hear the conversation from the evaluators a little bit later. At this point, I want just deal with a couple of questions that I have come in. One question is can the collaborating agencies be a combination of non profit and government agencies. The answer to that is a definite yes. This program announcement envisions a collaborative applying that is -- that includes at least two agencies and organizations, one of which is the lead agency, and all forms of organizations are eligible applicants including a (poor) profit firms if they agree to weigh their fee and profit. So that includes public agencies, private non-profits and private poor profit firms. The second question, we have read these in the FAQ is but it's still a bit confused as to the co-applicant versus supporting partner designation. In our situation, we are the applicant and will be administering funds in employing staff. One of our partners will provide some technical assistance and we will use grant funds to compensate them for doing this. All the other partners are providing services to the program free of charge. Some of the services, they are providing include clients training and counseling. Others are simply cross referrals. If everyone a co-applicant for just a partner that will receive funds from the grant. That's really a local option thing as long as you have at least two and you certainly should have whoever funds would be flowing out to in terms of your partner included as co-applicant. The others would probably strengthen -- the other partners would strengthen your application probably but are not absolutely required. I believe there are also at least one more question that I'm aware of. If there a preference of funding to be spent on direct service delivery or coordinating systems of care. The answer to this question is -- is probably yes. There is a preference on direct service delivery, hence the title promising approaches. And if you read the program announcement in some detail, you all notice that it is very evaluation orientated. That is we are interested at the child level outcomes and coordinating systems of care, would be terms of care would be much more difficulty to evaluate in terms of how they affectindividual children. So there is a preference, I think, would be safe the say. Okay at this point, I think unless one of my colleagues has anything else to add, I think we want to open it up for the general questions and answers on the program announcement itself. So if I could you ask the operator?

Operator: Thank you. Ladies and gentlemen, if you have a question at this time, please press the "1" key on your touch-tone telephone. If your question has been answered or you wish to remove yourself from the queue please, press the "#" key. One moment, our first question comes from (Larry Murrie) of National Center for Addiction Substance Abuse. Please go ahead.

Larry Murrie: I have to just clarify that's on Addiction and Substance Abuse. Yeah, I am curious as to the data collection requirements of the announcement. They do not seem to be that strenuous, so I was wondering if -- are there any types of methodologies that you are looking for us to employ or is it --

Bill Schechter: yeah

Larry Murrie: -- is towards to figure out the best way to collect.

Bill Schechter: Could -- could I ask you to differ that question for just a few minutes because we want to deal with the general questions around the Program Announcement and then we are going to have a brief presentation on the data collection aspects and then open it up for questions and answers on data collection evaluation. So if you wouldn't mind getting back in the queue when that time comes. Would that be all right? I'm going to take silence as a yes.

Operator: Our next question comes from Patricia (Stern), please go ahead.

Patricia Stern Hi, I just wanted to clarify what you are saying about at least two agencies, are you saying that at least two agencies need to receive funding?

Bill Schechter: No, not necessarily. At least two agencies need to be involved in the collaborative. One of those agencies would be the lead agency that actually receives the funding, whether or not the second, third, fourth, fifth receive funding as a matter for local option.

Patricia Stern But they would need -- I mean is it okay if the other agencies are providing referrals and the one lead agency is providing services.

Bill Schechter: Yes.

Patricia Stern Yes.

Operator: Thank you. our next question comes from Ellen Marshall, please go ahead.

Ellen Marshall: A (indiscernible) to the previous question, you just talked about relating to partner and so if we do have, you know, three or four other partners who aren't receiving funds, how will we list them in the application?

Bill Schechter: As you saw in the requirements, it said that you -- the application must come from a collaborative of at least two agencies, one of which is the applicant and the others are the co-applicants. They would be listed as co-applicants and in the evidence of collaboration, whether it's a memorandum of understanding or letter of agreement or whatever local practice calls for, there would have to be clear evidence of their role and commitment to the purpose of the demonstration.

Ellen Marshall: Okay, thanks.

Operator: Our next question comes from April Sharp. Please go ahead.

April Sharp: One of the jurisdictions in my State is already part of the phase one demonstration project. Will you consider funding another jurisdiction in the same state for phase two?

Bill Schechter: Yes. There be -- there would be consideration however, if you recall later in the program announcement, it also says that geographic considerations can be considered on final award. So, there is nothing that prohibits you from applying or being considered.

April Sharp: Just to add on to that, the jurisdiction in our state that receives funding is the largest metropolitan jurisdiction in the state and we are from a small remote rural jurisdiction.

Bill Schechter: Yes, I understand. I think you probably wrote in an e-mail about that.

April Sharp: Yes, I did.

Bill Schechter: Right. And -- and that's a good example of why we don't want to have any set rules like saying no because they are two different demographic -- two different sets of demographics and two different sets of -- of ways of approaching the problem.

April Sharp: Thank you.

Operator: Thank you. Our next question comes from Shereen Bleu, please go ahead.

Shereen Bleu: I understand that each co-applicant or collaborative partner must submit letter of commitment individually. We also talked about memorandums of understanding and is that related to the co-applicant partners or is that with other outside agencies?

Bill Schechter: I think what we are looking for in the response is evidence of a collaborative being in place with agencies committed to doing something not just supporting the application.

Shereen Bleu: So that would be the letter of commitment or the memorandum of understanding?

Bill Schechter: It's -- the problem is that different jurisdictions attach different meanings to those things and so either a memorandum of understanding or a letter of agreement that stipulates the relationship and the responsibilities of the parties would meet that. There -- there -- I would refer you also to the frequently asked questions, I think there is more detail -- a more detailed answer in there on that point.

Katherine Darke Schmitt: Laurie, this is Katherine Dark-Schmidt, can you tell me how many questioners we have in the queue at this point?

Operator: Yes, we have four more participants.

Katherine Darke Schmitt: All right. At this point, we are going to defer those four questions until our next question and answer period and we are going to move on to the discussion of data collection and evaluation. If you are holding with a program question, we are going to open up the last 30 minutes of the call to trying to clear the rest of the question queue. If you have a question that is not addressed today because we run out of time or if questions come up for you after this conference call, you are welcome to call the program staffs here at OJJDP at 202-307-1341. Now, that is to ask questions about the technical requirements of the program announcement. We are not allowed to comment on any ideas you have or plans to propose because we can't assist any applicant over another but we can address questions about what is technically required. I am going to move on now to begin to talk about the data collection part of the Safestart Promising Approaches evaluation and I know there are several of you who have questions about how this will work. As described in the program announcement, there is no local evaluation or research component of Safestart Promising Approaches. The local site do not need to have an evaluator on staff, do not need to have an evaluation plan and will not be involved in any kind of evaluation design, which is specifically -- to themselves. Safestart promising approaches will be the focus of a national evaluation which OJJDP will fund through a separate competitive process which will resemble in some ways the same process the sites will go through this fall. That national evaluator will have the responsibility of producing the design to evaluate the sites and for working with the sites to implement the evaluation design. Now, this include work with the comparison or control group. You will see in the program announcement and in the frequently asked questions documents that, there is a requirement that sites include in their applications some suggestions or how a comparison or a control group might be drawn from their community. We are asking only that you give us an idea of what kind of similar populations might be available to us from which to draw a comparison or control group. The sites will have no responsibility. We are recruiting participants from the Comparison or control group or from collecting data for us from those participants in the control or comparison group. The sites only data collection responsibility is to collect the data from the children served or from proxies where proxy measures are more appropriate and the variables that must be collected are detailed in the program announcement. We have asked that you consider and submit in your application, some ideas for how a comparison or a randomized no treatment control group may be used. We are aware that there are some jurisdictions which for political reasons may not be able to adopt a random no-treatment design. We are aware that, that may come up in your community and if that is the case in the jurisdiction from which you are applying then in the application you need to note the reasons why a no-treatment randomized control is not possible for your applications and you must supply information about how we could use a comparison group in your site instead. A pre-post test of treatment to children is not an acceptable alternative. There must either be a comparison group or control group. But as I said, you need only make suggestions about from where this populations may be drawn. We have started getting the question, will applications that include proposal for a no-treatment randomized control group receive preference over those with a comparison group. Methodologically speaking, evaluation standards show that control group designs are stronger than comparison group design. So, all other things being equal there would be a slight preference for a site that can produce a randomized design. However, I am expecting that perhaps as many as half of the applicants will not be able to pursue a randomized design and that will not in any way disqualify your application from consideration. The national evaluator which is mentioned in the program announcement in the FAQ has not been selected. They will be selected through a competitive process during the winter. The national evaluator will design the national evaluation which may include either local evaluators working very geographically close to you an data collection or perhaps some model that relies on staff from a central location flying out to visit you as necessary. We won't know what the national design is until the successful candidate for the national evaluator has been selected. If a national evaluator -- if a local evaluator design is proposed and selected then the sites will be asked for their inputs about how to identify and select a local evaluator. Sites will have access to the data collected from them and from their comparison or control groups for their own use. Should they wanted, I will remind you that any local evaluation activities that a site has to planned must not impede the national evaluation in anyway. If you submit an application which includes plans for a local evaluation, the reviewers will very carefully look to make sure that none of your local evaluation plans could possibly bias or impede the national evaluation plan. So please think very carefully before you decide to include local evaluation plans. The $10,000 set aside per year for data collection is to be used for the purpose of collecting data from the children served from intake at six month intervals until the end of the four year program. That $10,000 can be spend to hire a consultant, who could be a graduate student or could be someone on staff locally or could be a consultant identified from other source. The responsibility of that person will be to get data from all the children served, however, the site has plans to do that, having case workers or clinicians to administer the instruments or through whatever other method. Your data collector is responsible for compiling all that, putting it in a spread sheet and sending the spread sheet of raw scrubbed data to OJJDP. From OJJDP, the national evaluator will pick up the local site data and will use it in the evaluation. I am going to now address some questions that were e-mailed ahead of the teleconference. To begin with, there was a question asking about the program announcement requirements, that data collection instruments be validated and nationally known. We are not permitted to assist applicants in designing a strong evaluation, so it is up to the applicant to identify what kind of instruments would be best for the purpose as plan and in fact your selection of instruments will help us understand how well you know the field and what tools are out there. We have however, put together some materials that may be useful to you and you will see those at this web-site, the url is www.capacitybuilding, those two words no space, dot net. At that site, in addition to a lot of evaluation resources about the Safestart phase one evaluation, you will see a section on data collection instruments that may be useful to you. All right, those are the end of the evaluation and data collection questions that were e-mailed ahead of time. I will now open this up Laurie please, to ten minutes of questions specifically on data collection and any questions that you have about the plans for the national evaluation.

Operator: Thank you. Ladies and gentlemen, if you do have a question, please press the "1" key at this time. If your question has been answered or you wish to remove yourself from the queue, press the "#" key. We have a question from Reema Bhatia. Please go ahead.

Reema Bhatia: Hi I am calling from Edward Center in California. You said that one of the requirements fro the data collection person is to compile the data into a spread sheet. Will the evaluator be giving us this tool or can we budget for probably for a Microsoft Access tool that will help us compile the raw data and scrub it.

Katherine Darke Schmitt: For the purposes of your application, please do go ahead and decide which package you will be using to submit the data. It is possible that the national evaluator might ask you to tweak your format based on how other sites are submitting their data, so we can streamline this as much as possible.

Reema Bhatia: Thanks.

Katherine Darke Schmitt: You are very welcome.

Operator: Our next question comes from (Larry Murrie)./ Please go ahead.

Larry Murrie: How large a treatment group will you consider?

Katherine Darke Schmitt: Larry, excellent question. We have answered this a few times. We have explicitly left out any mention of a minimum treatment group in the program announcement and the purpose for not being specific about that is that we are expecting that we will be receive applications both from very small rural jurisdictions, or for example from tribal governments, where the base population is small and from urban centers where you are seeing hundreds of kids a day come into your ED. We will not select applicants on the basis of the size of the group of children they anticipate treating. That is only one of a large number of factors that will be important to us and we will be giving strong consideration to applicants from smaller geographic -- population density sites. So what the reviewers will be looking for is to see that the scope duration and intensity of the intervention you have plan, matches well, with your budget and the number of children you propose to serve.

Larry Murrie: Got it.

Katherine Darke Schmitt: Thank you.

Operator: Our next question comes from Evina Montanez, please go ahead.

Evina Montanez: Good morning, I have a couple of question and my first question is about instrument. The instrument that we are going to identifying our application, are thoseto be that instruments that we are going to use or other national evaluator going to make that final determination?

Katherine Darke Schmitt: For the purposes of your application assume that the instruments that you propose --

Evina Montanez: Uh-huh.

Katherine Darke Schmitt: -- will be the one you use.

Evina Montanez: Uh-huh.

Katherine Darke Schmitt: If for some reason, the reviewers were concerned that you had not selected exactly the right instrument or if the evaluator felt that there were a better instrument that would be just as easy and available to you, it is possible that OJJDP would come back and negotiate with the successful sites about what instruments will be used.

Evina Montanez: Okay, thank you. And my next question is about data collection manager, for instance if we have doctorial student who is going to collect our data, is a preference that he or she must be data collection manager or somebody inside from our organization?

Katherine Darke Schmitt: That is entirely a question of preference to you --

Evina Montanez: Okay.

Katherine Darke Schmitt: OJJDP has no preference.

Evina Montanez: Thank you.

Operator: Our next question comes from Sally (Bordam), please go ahead.

Sally Bordam: Yes, I also have a question about the instruments and excuse me a piece of that was answered in terms of the numbers. But my question is about, do we need to include a refusal to participate -- well if we would -- we would include a refusal for -- to -- a refusal to participate option, in administering any instrument. If someone refuses to participate, can we still include them in our Safestart overall data or way -- would they be excluded from receiving Safestart services. Could only the people who receives Safestart's services, would that only be able to be people who upon which we were able to utilize validated instrument whatever that is, that we select.

Katherine Darke Schmitt: I think that is a mater of site preference. The reviewers will be looking in your application for an understanding that the purpose of this program is to collect outcome data on children and they will be looking for the -- the strength of the case you make that you will be able to do that.

Sally Bordam: Okay. But I am not sure that -- that really answers my question now.

Katherine Darke Schmitt: Let -- let me try to work around it in another way. The reviewers are looking to select programs which can provide the best outcome collection plans possible, so if you've a strong data collection plan and over the course of data collection, you have some refusals which is not unusual, I am -- I am sure that the reviewers would perceive that, that is part of the course of doing this kind of research and would not be concerned about it. If you explicitly planned in your application to deliver services to people from whom you could not or would not collect data, then your application would look substantially different from the other applications that come in.

Sally Bordam: Thank you.

Operator: Our next question comes from Patricia Stern, please go ahead.

Patricia Stern Hi, do you want a copy of the assessment measures, we will be using or is a citation sufficient?

Katherine Darke Schmitt: A citation is sufficient, thank you.

Patricia Stern And so -- and do you want people to be assessed every six months even if they are currently in treatment?

Katherine Darke Schmitt: Unless there is clinically a reason why that is harmful or not desirable, the blanket answer is yes. If this is one of those fine points that needs to be negotiated after the award is made, then we can do that.

Patricia Stern Okay. Thank you.

Operator: Our next question from Lark Cirius, please go ahead.

Lark Cirius: Hi, I -- I would like to get some clarification what exactly is the data that the partner agency will have to collect. Can you give some specific examples or -- or refer me to somewhere where I can get that information?

Katherine Darke Schmitt: This is on page -- starts on page 38 through 40 of the Program Announcement list, the variables thing must be collected by the collaborative.

Lark Cirius: Okay. Thank You

Operator: I am showing no further questions.

Katherine Darke Schmitt: Excellent. At this point, we are going to move on to the presentation by the National Evaluation team of Safestart Phase One. As Bill Schechter: mentioned in his introduction to this call, please keep in mind that Safestart Phase One is really different from Safestart Promising Approaches. Nonetheless, the folks who have been involved in the evaluation of Safestart have some valuable things to talk about in terms of how implementing these services and how collecting data about the recipients of these services has worked in the Safestart site. We have two firms represented on the National Evaluation team and they will speak in turn each for 10 to 15 minutes and then we will open this up for questions again at the end of the second presentation. So Lisa Lunghofer is ready, we will begin with you.

example, would working groups be preferable to one small management body? What decision making process will you use? Specifically, who will you involve in that process? How are you managed several systems with separate agendas simultaneously. This can obviously be tricky and something important to think about at the outset of your program. Second, getting everyone on the same page really does help. In particular, training among service providers and collaborative representatives is important early on. We found that it increases the awareness of salient issues and the commitment to the process. Third, never underestimate the value of time. You need time to let the process unfold and this may seem obvious but I think for some times more optimistic than is warranted about how long things will take. Fourth, I mean -- I think this goes along with the importance of being realistic about time, you should also think about setting resources aside for the unexpected. And finally with respect to the -- the planning phase for this the phase one initiative we found that it was important for the grant sites to stay focused in scope. Kate, do you have anything to add about the -- the planning process?

Kate Abbruzzese: Sure. This is Kate Abbruzzese from the Caliber Associates working with lead (indiscernible). Couple of points, I just want to just reiterate that Lisa spoke to. In particular, it have to do with representation in the collaborative -- with the table. But one thing that we have walked away with in our experience with (indiscernible), we are anticipating certain changes . So if in your leadership you are anticipating a change, be it due to a politically appointed individual who is -- those are things to anticipate being able to anticipate has helped the transition through the way a little bit smoother. Another point I liked to reinforces is in terms of the structure for getting the work accomplished. That in addition to -- to making some decision around working group for small management bodies also to be aware and sensitive to the fact that at -- at points, there is in implantation of this rule may likely change and shift as -- as the point of implementation does. And then finally another point that I like to emphasize again is around the getting everyone on the same page helps in the training. Training early to really basically on assumptions is very helpful. They can be building an existing training from Caliber Partners and it may in some cases exclusive violence trailer training that has been developed as a result of the approaches that have been showed and so that really assists earlier on instead of facing some of the dilemma later in time of implementation. And that's -- those are few points that I just like to re-emphasize.

Lisa Lunghofer: Great thank you Kate. Next I would like to talk briefly about some of the lessons learnt with respect to implementation. First, it's important to think about what barriers you expect to experience in identifying, assessing, referring and treating your intervention children and families. It's really important to anticipate these early. What we found in the first phase of Safestart is that there should be no assumptions that if you build, they will come. You -- you need to think about your community in relation to the potential barriers and facilitators to engaging children and families in your programs. For example, rural communities may experience different barriers and facilitators from those experienced by urban communities. Second, try to anticipate challenges to finding qualified personnel to match program components for certain interventions. And then finally and I admit this is a big one. Examine where strategies fit in the context of each organization involved and how differing agendas, priorities and philosophies may affect implementation. As I said, that's -- that's a -- a big thing to do but I think that it's really important to think about those at the outset of your program implementation so that there are not points that you have tripped upon down the road. Kate, do you have anything to add to that?

Kate Abbruzzese: Now, there is just a couple of points that I could build on that you have mentioned. Again in relation to the barriers, to anticipate barriers and identifying, assessing, referring and treating children and families. Things to consider that have come up as challenges to some in implementations phase one are -- are different ways and innovative ways in identifying and providing services for families already receiving service as opposed to those not previously identified. So those approaches may require innovative consideration. In relation to additional barriers, it -- it is the case and -- and sometimes that's even factors such simple as meeting time and logistics will look different in urban centers as opposed to rural settings. So certainly planning for those when planning issues around meeting for implementation. I mean finally I would add that in -- in regards to finding qualified personnel to match program components for certain intervention. To really think critically about perhaps folks that are going to be working with the data are they used to simply providing services and working at a minimum level with data entry and -- and consistency. Consider their comfort level and experience, I was doing well. And that's all for now.

Lisa Lunghofer: Great. I think -- I think that's it. Katherine, we will turn it back over to you.

Katherine Darke Schmitt: Thank you very much Lisa and Kate. David when you are ready. David Chavis, Association for Study of Community. Study Internal Community, we are ready when you are. Okay while we are trying to find David, Laurie, can we open the queue here to take a few questions until someone locates him.

Operator: Yes, and I am still showing his line connected.

Katherine Darke Schmitt: Okay.

Operator: If you do have a question, please press the "1" key at this time.

David Chavis: Hello, oh.

Katherine Darke Schmitt: Oh David --

David Chavis: Sorry.

Katherine Darke Schmitt: -- there he was.

David Chavis: There was -- I am sorry I apologize. There was something wrong with my phone and so --. Anyhow --.

Katherine Darke Schmitt: Laurie, we are going to close the question queue then. Sorry for the false alarm folks, this is David Chavis of the Association for the Study and Development of Community.

David Chavis: Good -- good afternoon everyone. I am going to talk about three issues that we learned from providing assistance and -- and collecting the data locally during the phase one Safestart program. The first set of issues around internal capacity to collect the data, the second will be the preparation for assessment, data collection and management and the third is developing comparison groups. Basically, the theme here that I am going to talk about is it's better to pay now than to pay later as we have learned over the last several years. In terms of internal capacity to collect data, we found that data collection methods and systems should be within the existing capacity both in terms of you mean in technological of the applicants and your partners. If not, you need to plan for appropriate technical assistance and that can be from universities or from private vendors. These Safestart data collection procedures should built upon existing data and other information collection systems, in order to minimize the effort, confusion and complications. Many sites spend a lot of time developing new systems and that drew a lot -- drew a lot of attention and resources. And several of the sites they have actually been able to develop common MIS systems that have effectively linked them with each other and through the conversations around developing -- addressing issues of confidentiality, they've been able to strengthen their collaboration, in the process of developing that programs. It was also recommended in regards to an earlier conversation, that people use off-the-shelf software like Microsoft Access or other things that are commonly available so that their partners are able to get them and participate in the data sharing at a lower cost. That receiving the consent of families to participate in the research has been a major challenge and issue for the current Safestart sites. And that one of the biggest challenges that has been to be able to address the needs of culturally in terms of economic and ethnic diverse groups. And so that sensitive procedures for recoupment and form consent and -- and continued engagement with these groups need to be developed. Groups that -- or that some of sites they have found that they can develop a bilingual -- need to the bilingual consent forms, work with other agencies who may not be in the mental health or service areas, but have close ties to ethnic and to low income communities. So if they can engage them more effectively into work as -- work with them as part of their partnership. Line staff need to be brought in the data collection process as well as management and steps need to make sure that -- be taken that the line staff see the values of data collection in their work and have the time and capacity to participate in the data collection process. This is one of the things that we need to emphasis the most because it found that, while management and everyone sitting at meetings set the data collection requirements are -- are great idea, the line staff don't see the value of them. And so there has been a lot of resistance to the extra data responsibilities and so that it's important to spend time up front, talking about how evaluator not only for participation nationally, but how that information can be used to improve their practice and services to these families. The second groups of lessons learned has been the preparation for assessment and data collection and management. We -- groups have found that it's important to be able to really thoroughly review, and advance research on assessment and instruments in order to later problems. That they need to check on the length of the time, that it takes to administer it, the skills needed by an assessor, the age and culturally appropriate groups and validity of it, for use with particular populations related to exposure to violence in children. That group has been able to use local academic and other expertise in order to help the site. I am going to put another plug in for our web site at www.capacitybuilding.net. Where there is a button, on it for the Safestart program and then if you look under the listing on the left hand side on the measures, there is a chart of measures that have been found in the different groups that they have been found, work with in the amount of -- of the effort that needs to able to administer them. Also we need to, if you are working with contractor and different mental health professionals, their -- their capacity to be able to do assessment, their willingness to include assessment as part of their practice and to include it in all sorts of contractual relationships is very important and if not again TA is needed to be provided to them in order to how use these assessment tools before committing to it. Several -- many of the sites had difficulty training contractors due to time and resources limits -- limitations. And it's important to make sure -- when you are talking to contractors of providing services that they are -- they are very clearly understand these expectations. Grant issues to be able -- should have a clear understanding of the partner agencies, data capacities, not all agencies participating have the same amount of data collection capacities and time lines so that they needed to be able to make these expectations well understood. Many of the sites had a collaborative approach, completing the initial RFP and that being involved in responding to this program announcement together and really discussing before they make a commitments, what capacities are needed and what needs to be -- to strengthened. Sites need to have to be able to get IRB, Institutional Review Board Approval to share data, as well as HIPAA and that different kind of confidentiality and institutional requirements should be looked at earlier rather than later. The final group of lessons learned relatively quickly or has been around the comparison group and again, control groups. The control groups as Katherine has said, are those that are from the same population that are randomly assigned by either treatment or control groups. Many of the sites should at least -- let me talk a little bit more about the comparison groups where they were looking for similar population to be able to look at -- at the changes and then in order to compare to the groups and treatment that we found based on basic scientific rules that I should at least attempt to recruit at least 30 members for their comparison and control groups in order to deal with any kind of a attrition as well as having adequate number. One idea is to work with community agencies that are serving a similar population exposed to the same form of violence in order to develop comparison and control groups. In some cases, some of the sites have been able to identify families that are in the waiting list for services with similar exposures, some sits have worked with headstart and similar populations who are not receiving treatments to engage them. They can also look at other communities that have other communities and neighboring communities with similar populations whether unserved neighborhoods, that serves similar populations in order to use them in the comparison groups and then finally, early on that in the process that grantees need to be able to develop a plan and strategies for retaining the comparison group over the life of the study such as collecting contact information from at least two closest family and friends. Thank you.

Katherine Darke Schmitt: Thank you David. Laurie, at this time, we are ready to open the queue for questions about the evaluation and presentation and any further questions that people wish to have addressed during the conference.

Operator: Thank you. If you have a question at this time, please press the "1" key on your touch-tone phone. We have a question from Reema Bhatia, please go ahead.

Reema Bhatia: Hi, I actually have two questions and I believe it is supposed to be evaluation section. First of all, I was wondering are you at all interested in our collaboration working on any dissemination efforts of our program design and also looking at policy and system change on the local level as been seen in phase one so.

David Chavis: The answer is yes, but the primary purpose is to -- to test which approach is worked best where and what are the contributing factors. So those are secondary considerations which really important and that is why there's been so much attention to the evaluation data collection site of this is that we want to see -- what -- we want to be able to know the effectiveness of the various approaches that have already been developed in other places and that you all might adapt in your own situation.

Reema Bhatia: Great. That actually leads into my next question. We have been able to engage as a partner in organization that was listed in your RFP that could help share a program design with us and we were adopted in at completely different municipality for slightly different target population but that partner was also included in a collaborative, that was huge funding in phase one, will that be a problem they would get some training, money through our grant as we have planned right now.

David Chavis: Uh-huh.

Reema Bhatia: They are not a lead agency but they are actively involved in the other -- in a different location.

David Chavis: I think that's a -- the kind of question that we better do some caucusing on internally. I -- I don't have an off the top answer for that. It -- it's certainly raises a question.

Reema Bhatia: Uh-huh.

Reema Bhatia: On the one hand, we don't -- we want to encourage our existing sites to be advocates for what they -- what they have done and what they have learned but part of the exclusion of those sites from this program announcement was to spread the notion of Safestart and the various approaches in other areas across the country.

Reema Bhatia: Excellent.

Katherine Darke Schmitt: Could I have you do this, do have a pen?

Reema Bhatia: Yes.

Katherine Darke Schmitt: Could you please call this number and leave us contact information for you so that after we had a chance to discuss the implications of your questions we can contact you personally.

Reema Bhatia: That would be great.

Katherine Darke Schmitt: This is the phone number 202-307-1341.

Reema Bhatia: Great and I can give some more details on who we are talking about.

Katherine Darke Schmitt: Thank you. Bill, is that acceptable?

Bill Schechter: Yeah, that's certainly.

Reema Bhatia: Great.

Katherine Darke Schmitt: Okay. Thanks Laurie, we are ready for the next question.

Operator: Thank you. Our next question comes from Susan Lindall, please go ahead.

Susan Lindall: Hi Katherine. How are you? Just one question. And what David -- what David was saying, they kind of tipped it out anther question that I had regarding long term follow up -- the 24 hour month follow up with the children and I don't think this is relevant but in terms of incentive for the long term follow up. Do we have a responsibility for continuing to make contact with the children served.

Katherine Darke Schmitt: You have responsibility for continuing to make contact serve with the children who are initially part of your program --

Susan Lindall: Huh-uh.

Katherine Darke Schmitt: -- for the duration of the four years, and if you chose to build incentive structure into your budget to do that that's fine with us.

Susan Lindall: Okay. And there are going to be I would assume that the national evaluers are going to be providing the tracking and the locating of families over the two year follow up period?

Katherine Darke Schmitt: The national evaluator will track and follow the comparison or control group --

Susan Lindall: Right.

Katherine Darke Schmitt: -- will not follow the kids serve.

Susan Lindall: Okay -- okay, that answers it.

Katherine Darke Schmitt: Okay.

Susan Lindall: Thanks.

Operator: Our next question comes from (Larry Murrie). Please go ahead.

Larry Murrie: Right. This sort of relates to the last question, so thank you. The apple kids or the groups awarded the contracts will not be responsible for doing anything of the control group. Is that correct?

Katherine Darke Schmitt: That's correct Larry. All we need you to do is make a suggestion in your application about how those groups might be chosen and then to help inform the national evaluator as the national evaluator makes plans to actual to actually make do the contacting and data of collection from that group.

Larry Murrie: Good. And just to clarify the treatment group that's in the first year will be followed up for a couple of years that does not preclude new youngsters being admitted to the program and new youngsters and families, does it?

Katherine Darke Schmitt: Absolutely.

Bill Schechter: Absolutely not.

Katherine Darke Schmitt: You -- you can continue to admit through the four year duration of the program.

Larry Murrie: Right. My last question, I think, I hope, I didn't know that the planning period built into this application, is there one?

Katherine Darke Schmitt: No.

Bill Schechter: No, there is not. The notion here and this is one of the differences between phase one and phase two is that this is a way to test particular program approaches that people have already been working on in their communities or want to adapt from what somebody else has done elsewhere in the country and so the only time to be doing the planning for this is before the grant award.

Larry Murrie: I understand.

Bill Schechter: Not afterwards.

Larry Murrie: That's very helpful.

Operator: Our next question comes from Patricia Stern. Please go ahead.

Patricia Stern I had, a just a question about IRB clearance and what do you mean by that. Do we need to apply to our IRB before receiving the grant or just get some kind of letter from our IRB saying they will allow us do it if we were awarded?

Katherine Darke Schmitt: Those are fine, Patricia. Yes, as option.

Bill Schechter: What's an IRB?

Patricia Stern: Oh In--Institutional Review Board.

Katherine Darke Schmitt: Thank you. Patricia, we would like to spell out the acronym in case there are folks on the phone who are not as fluent with those the way we abbreviate things. So folks the answer to that question was you need not have Institutional Review Board's clearance at the time your application is submitted but you must have submitted it, your project to a board and be able to tell us when the review will be concluded.

Operator: Our next question comes from Caroline Thomas.

Caroline Thomas: Good morning. I am calling from Northern California and our program has actually been up in running for the last two to three years and we have been working with several agencies across California for our programs and the sort of the nexus of our program is an educational piece, a training piece and we have developed a training curriculum. Do we need to include if -- if that is in excess of our program, do we need to include that curriculum as part of our proposal?

Katherine Darke Schmitt: Bill.

Bill Schechter: Yes, I would say you would because one of the things the reviewers are going to be looking at especially with approaches like that is what kind of variety are you talking about. I would caution you however to think about how the training strategy that you are discussing will fit in with the performance measures and case variables that you would be required to participate in as part of this, especially the case variables.

Operator: Our next question comes from Robert Nelson.

Robert Nelson: Earlier, it was stated that you were looking for more geographically -- geographically stated programs and in phase one we are about approximately 90 miles and about the same size city of one of your phase one sites. It's -- is that leading me to -- to discourage us from applying, is that a fair statement?

Katherine Darke Schmitt: No.

Bill Schechter: I -- I wouldn't -- I wouldn't consider it discouragement from applying. I mean the reviewer is going to look at the strongest and -- and -- and rate that proposals on their strength. However as you did see in the program announcement when final selection time comes, geographic considerations are among those that can be utilized. So it's not a discouragement.

Robert Nelson: Thank you.

Operator: I am sorry no further question. Actually, we do have a question from (Lark Cirius). Please go ahead.

Lark Cirius: Hi. Well -- well my name is not Serious, (Cirius). I am not very serious actually.

Bill Schechter: But you are from Elgin, Illinois.

Lark Cirius: Yes I am from Elgin. I need some clarification, -- I have never ever heard of Safestart but for about few weeks ago, so I am really trying to catch up here with my education. Could you please clarify all these phases. I am hearing phase one, I am hearing phase two. Is it you know, is there phase three, the phase four. What are we talking about exactly?

Katherine Darke Schmitt: Bill, could you --?

Bill Schechter: Yeah, phase one and phase two are just ways that -- that we have chosen to designate the program announcements and grantees funded and how they differed. This is not a sort of a annually competed program appropriated for by Congress. This is -- this is a -- a demonstration that had one phase, phase one with the eleven sites around the country, that was looking at one set of things. Now, phase two which is coming along later actually five almost six years after the first one was announced is to -- to see what we can learn about what works where. Now, whether there will be future phases is certainly not -- not a issue that's been determined yet. So that's all we mean by phase one and phase two.

Lark Cirius: But what we are applying for it right now is in phase two.

Bill Schechter: Exactly.

Lark Cirius: Okay. The -- okay.

Bill Schechter: Don't -- don't think of it in terms of project phases like you would do --

Lark Cirius: Okay.

Bill Schechter: -- you know, locally.

Lark Cirius: Okay, thank you.

Bill Schechter: Sure.

Operator: Our next question comes from Sally (Bordam).

Sally Bordam: Yes. I have a question in response to what you just said about IRB clearance. You said that our project must have been submitted to an IRB at the time of application and I guess that was a little bit different than how I read the RFP. And I am not sure how to do that without additional information about the control and comparison group at the time of submission.

Katherine Darke Schmitt: Sally, I can't help you negotiate the individual course of your particular IRB. I know each one of them has a different stance on at what point they will review a project and what level of review they are willing to give it at what point. We need to see evidence in your application that there is an institutional review board review plan. If --

Sally Bordam: But, go ahead.

Katherine Darke Schmitt: No, that's all right. Go ahead -- you go ahead.

Sally Bordam: But it doesn't have to have actually been submitted to that IRB at that phase or it does?

Katherine Darke Schmitt: Well, we need to know that there is an IRB who is available and ready to clear your project. I -- all right, I will withdraw my use of the term submit earlier. It need not have been formally submitted. So, we need to know who that IRB is --

Sally Bordam: Uh-huh.

Katherine Darke Schmitt: -- and when you will be submitting. What details is it that you are waiting for?

Female Speaker: Okay.

Katherine Darke Schmitt: And I would suggest that you -- if there is any sort of expedited or preliminary review that is available I think the stronger applications will have some evidence that some -- that negotiation with the IRB is at least underway at the time of the application.

Operator: Thank you. Our next question comes from Carol Sedenko.

Carol Sedenko: I have three questions. One is, I am assuming that it's acceptable for the lead organization to be a private non profit foundation and the second question is, in regard to the comment about how the training will relate to the training strategy will fit in with the performance measures and case variables, could you be more specific in what you mean about case variable?

Katherine Darke Schmitt: Bill?

Bill Schechter:: The answer to your first question is, yes absolutely. It certainly an eligible organization. There was an earlier question about what are the kinds of information that need to be collected on each child served by this -- by whatever your promising approach is. And Katherine pointed to you to the appendix in the program announcement which runs from 38 pages -- 38 through 40 and you will see there are a number of what are called case variables there. That's the kind of information that need to be collected. Only one of those that I see which is like the third one down on the page has to do with the training things. And so if -- all your promising approach does is train people, it will be difficult to evaluate your approach in terms of it's impact on child outcomes. Does that make sense?

Carol Sedenko: Yes. Very much so, thank you.

Operator: We have a follow up question from (Lark Cirius). Please go ahead.

Lark Cirius: Actually it's not me. It's my boss this time.

Female Speaker: My questions is there is another site geographical -- there is another site that is in a different county but it's relatively close to us geographically. And I am not sure what the population it serves or how that affect us in our application?

Bill Schechter:: You are -- you are in different jurisdictions. You are in Elgin which is what Kane county or something?

Female Speaker: Right.

Bill Schechter:: Okay. That's and it's the city of Chicago as our existing jurisdiction and so the demographics obviously are different and the geography is different. So, it would not rule you out.

Female Speaker: Okay. We understand you have on the Streamwood too.

Bill Schechter:: In where?

Female Speaker: Streamwood, Illinois.

Bill Schechter:: I don't believe so. Not that I am aware of.

Katherine Darke Schmitt: No. Definitely not. We have encountered in the process of this program and also being on the street. Many jurisdictions which -- which have local programs -- programs which they happen to called Safestart.

Bill Schechter:: All right.

Katherine Darke Schmitt: But which are not actually related to this federal effort.

Lark Cirius: Oh, okay.

Female Speaker: Thank you.

Lark Cirius: Thank you.

Bill Schechter:: Yeah. The only jurisdictions that are not eligible to apply are the 11 listed in the program announcements or on any of the web-sites about this Safestart national demonstration.

Operator: Our next questions comes form April Sharp.

April Sharp: I -- I just want to -- this is back to the subject of IRB review and I just want to clarify that this is on 12 -- page 29 of the RFP, it speaks to single point of contact review and executive order 12-3-72. Is the same as the IRB?

Katherine Darke Schmitt: Yes it is. It is possible to get a single project clearance under federal law which would be of assistance to those organization which do not have in-house institutional review board.

April Sharp: I believe I just got more confused. I thought I was clear.

Bill Schechter:: No.

April Sharp: What --

Bill Schechter:: Are -- are you talking about the single point of contact review on page 29?

April Sharp: Yes.

Bill Schechter:: I think that has to do with -- with -- with coordination of who can apply for federal funding within given jurisdictions.

April Sharp: Okay.

Bill Schechter:: And I -- and I am not sure that deals directly with the institutional review board question.

Katherine Darke Schmitt: Yes. Bill, you are correct.

Bill Schechter:: Yeah. It has more to do with what the old days was called the Federal clearing house. Any -- any group that wanted to apply for federal funding within the jurisdiction of a city, county or state had to notify this clearing house that they were doing so and one of the purposes was obviously to coordinate the kinds of applications and federal support that was going on.

Female Speaker: Well, then --.

Male Speaker: That's what that deals with.

Female Speaker: Okay. Well, then I need a little more clarification with, what is an example of an in -house institutional review board.

Male Speaker: Oh, that's something totally different and Katherine will deal with that.

Katherine Darke Schmitt: This is discussed extensively in the fact, so I am going to point you back to that. In general --.

Female Speaker: Wait a minute, where are you pointing me, to the RFP

Katherine Darke Schmitt: Those frequently asked questions doc.

Female Speaker: Oh, okay, got you.

Katherine Darke Schmitt: In general, universities, private research firms, hospitals have institutional review boards already formed in operating because it's just part of their bread and butter. It's how they do their work everyday. These kinds of research studies. So if you have among your collaborative an agency like that, it's likely that you are not going to have to look very far --

Female Speaker: Oh.

Katherine Darke Schmitt: -- review board.

Female Speaker: Okay. But I would still as a -- have to also go through the single point of contact review as well.

Katherine Darke Schmitt: Yes and I am sorry. I should not have jumped into your question before I was sure I understood it. Bill Schechter:, a moment ago answered you correctly that single point of contact review has to do with how your state control applications for federal funding. It has nothing to do with us. You need to make sure that you have coordinated if that is the requirement in your state. The institutional review board is about federal law and it's completely different.

Female Speaker: Thank you very much for the clarification.

Operator: Our next question comes from Barbara (Humbles).

Barbara Humbles: I guess, I am still trying to follow with IRB as well and my -- and I was almost clarified with the last caller but I am still concerned about if you are not collaborating with an agency that has an internal one, where in your state do you contact to find one.

Katherine Darke Schmitt: I am going to refer -- refer you to the frequently asked question --.

Barbara Humbles: Okay.

Katherine Darke Schmitt: -- the document that makes some suggestions about where an institution that didn't have one might go to find one.

Barbara Humbles: Very good.

Operator: Once again, if you do have a question, please press the "1" key at this time. I am showing no questions.

Katherine Darke Schmitt: All right.

Operator: Actually, we do have one participant who just --

Katherine Darke Schmitt: Okay.

Bill Schechter:: (Indiscernible).

Operator: Ms. Shelly -- (Shelly Krope). Please go ahead.

Shelly Krope: Yes. There was this stipulation that about the 501C3 status or 502?

Katherine Darke Schmitt: Right. Shelly, I think I addressed your question -- oh, may be I missed you this morning when I tried to call. However, a number of other folks have asked this question so this is a good place to address it.

Shelly Krope: Uh-huh.

Katherine Darke Schmitt: As you register your organization on our grant management system web site, you build a profile for your organization by giving us some information about what kind of organization you are and who your contacts and authorities are and so forth. In the process of building that profile of your organization, we have had a few folks accidentally typed in 501C2 when they meant to type in 501C3.

Shelly Krope: Oh.

Katherine Darke Schmitt: If you had submit you would have now immortalized that so that every time you look at your information, it lists you as a 501C2. If this has happened to you we can go back and undo it. Do you have a pen, are you ready to write down our phone number?

Shelly Krope: Yes, ma'am.

Katherine Darke Schmitt: Okay. The technical guy on our staff who handles this is Mr. Rich Hull, H-U-L-L and you can reach him at 202-616-4506 and Rich can walk you through the process of revising your original registration data to take care of that. However if you didn't get to it, rest assured that we will know what you mean and it would not count against you that somebody's finger had slipped. We will understand that --.

Shelly Krope: Okay. I planned it that and I -- I am -- it's done correctly now but I don't know. I will call him anyway.

Katherine Darke Schmitt: Okay.

Shelly Krope: Thank you, very much.

Katherine Darke Schmitt: No problem.

Operator: We have a question from Reema Bhatia, please go ahead.

Reema Bhatia: I am calling from Edward Center in California, again. This is a question about following the follow-up data on the client research.

Katherine Darke Schmitt: Yes.

Reema Bhatia: You have been saying on the phone call that we need to track them for the full four years, but I thought I read in the RFP that we needed to track the data just for two years after their treatment. Which is correct?

Katherine Darke Schmitt: You need to track the data every six months for 24 months after intake, but for -- for (indiscernible), for classes of children that enter at year two, you will be tracking them through the end of year four. So for all four years, there will be some data collection going on. Each child needs 24 months of tracking at six months intervals.

Reema Bhatia: Okay, so 24 months after intake. So, if our treatment requires a year, there might just be a years worth or two sets of follow up data. Is that going to be acceptable or would you prefer that we extend it for a longer period of time?

Katherine Darke Schmitt: We would prefer that you are collecting post intervention data for 24 months.

Reema Bhatia: Okay. 24 months.

Bill Schechter:: As long as it falls within the --

Katherine Darke Schmitt: The four year period.

Bill Schechter:: -- 48 months period of the --.

Katherine Darke Schmitt: Yes.

Bill Schechter:: Yeah.

Reema Bhatia: Okay, great. And for the control group, if we are doing the randomized design, that would mean there are people for children and families from our agency who would not be served by a program but the national evaluator would still the contacting them to get data?

Katherine Darke Schmitt: That's correct. They would be administering the same instruments that you are using with your treatment children.

Reema Bhatia: But they would be administered by the evaluator rather than by our staff?

Katherine Darke Schmitt: Correct.

Reema Bhatia: Okay. Thank you.

Katherine Darke Schmitt: Thank you.

Operator: Our next question comes from Patricia Stern.

Patricia Stern: Hi, sorry. One last question. Just for the budget, do you want to budget for the first year or for all four years?

Katherine Darke Schmitt: Bill?

Bill Schechter:: Again, that's enumerated in the frequently asked questions but the budget you submit is for the first year only for the up to $210,000 amount. And then if you are successful, each year afterwards you would submit a new budget and work plan for that year.

Patricia Stern Right. Thank you.

Bill Schechter:: Sure.

Operator Our next question comes from Mary Harris.

Mary Harris: Yes, I have more of a -- a general program question for you. There's been a discussion with my colleagues in my office as -- obviously the -- the emphasis is on all the training and education and actually testing out new and better strategies for delivering services. We've had some discussion about really, are we able to use the funds to -- to actually provide the direct services. In other words, staff that are -- that are providing the services or is that not allowable or are you looking for the process more to be funded?

Katherine Darke Schmitt: Bill?

Bill Schechter:: No, it's -- it's definitely the former. We're -- we're very much interested in the delivery of services and the impact those services have on the children served or their families. So, that -- that's the primary consideration.

Mary Harris: Okay. So we could bill cost for direct service staff into that budget then?

Bill Schechter:: Oh, absolutely.

Mary Harris: Okay.

Bill Schechter:: Uh-huh.

Mary Harris: Thank you.

Operator: Once again, if you do have a question, please press the "1" key at this time. I showing no questions. No questions.

Katherine Darke Schmitt: Laurie, thank you so much for you help. Participants, thank you very much for your time this afternoon. Please note that we do have two future audio conferences scheduled. Presentations by the speakers will be the same but you are welcome to tune in and welcome to ask further questions. Those -- and the times of those calls are posted in the frequently asked question document. In addition, we will be posting the transcript for both this teleconference and the teleconference that was completed earlier this week. You will find directions in the frequently asked questions document for reading those transcripts. I am hoping that that information will be posted by the end of the day tomorrow.

Female Speaker: Well, Tuesday is on -- the teleconference is on the 17th. Today's date will be posted by close of business.

Katherine Darke Schmitt: Excellent. So, hope you can look at the frequently asked questions document today just -- the end of the day, just see the transcript from Tuesday's call. Give us a few more days to get the transcript from this call up. In addition you can listen to a recording of this call, 24 hours a day, seven days a week, up until the time of our next conference -- a teleconference which is I believe scheduled for next Wednesday. The frequently asked questions has directions for how to find out how to listen to the call. If you need other guidance from programs staff on the technical requirements of the program announcement, please call us. That number is 202-307-1341. Speakers, please stay on the line so we can be moved to private call to debrief and participants, thank you very much. This concludes the conference call.

Operator: Ladies and gentlemen, this concludes the conference. We thank you for your participation. You may now disconnect. Everyone have a great day.

AMBER Alert | National Sex Offender Public Registry | OJP Reentry Initiative

USA.gov | Privacy | Policies & Disclaimers | FOIA | Site Map | AskJJ
A component of the Office of Justice Programs, U.S. Department of Justice