Tania: Good day ladies and gentlemen and welcome to your 2004 Safe Start Promising Approaches call.

Bill Schechter: Thank you so much everyone for joining us. This is Bill Schechter from the office of Juvenile Justice and Delinquency Prevention. I want to just give you a brief overview of what we are going to try to accomplish in the -- in the next hour and a half. The first thing we will do is to have a brief presentation and questions and answers about the program announcement itself and then in general and then we will turn to the data collection and evaluation aspects of the program announcement. And deal with questions and answers on that and then we will move to the evaluators from the first phase of the Safe Start Demonstration project, the eleven existing sites, to share their lessons on what they have learnt about the program itself and data collection issues, evaluation issues and other pieces of information that might be useful as you are finishing up your proposals. First, let me introduce who is going to be on the call. On the call, we have Katherine Darke-Schmitt from here at OJJDP, who will be talking about data collection and evaluation issues. I have already introduced myself Bill Schechter. And I believe we will have Cecilia Duquela-Fuentes, another program officer here at the OJJDP. And then from the two evaluator firms for the first phase of Safe Start from Caliber Associates, we have Lisa Lunghofer, Janet Griffith, Susan Chibnall. And from ASDC, that stands for the Association for the Study and Development of Community, Inga James. At this point, I would like to remind everybody to have the 53 page program announcement and the most recent version of the frequently asked questions in front of them. I think that will make it much easier for everybody to deal with especially if we start talking page numbers. We are going to have, as I said before, brief presentations and then open it up for question and answer after each section. And at the end of the call -- at the end of the hour and a half call from 01:30 to 03:00 Eastern time we will have general questions, any that didn't get answered before. I would like to also remind you that if you ran out of time on this call to ask questions, the information number at OJJDP is 202-307-1341. And I encourage you to call and leave a message on that. Also you should be aware of the capable communities web-site which is www.capacitybuilding, one word, .net, which has a treasure trove of information about evaluation of the existing Safe Start demonstration as well as resources of general interest on evaluation of such projects. At this point,. I would like to remind you off several important points. This being the fourth and final of our four audio conferences.

Female Speaker: Uh-huh.

Bill Schechter: The deadline for registration, remember this is all online, the deadline for registration is midnight Eastern time September 8th, that's tomorrow, Wednesday. The deadline for applications is 08:00 pm Eastern time on Friday, September 10th. There are no exceptions. We anticipate with the incredible number of registrants so far, a very busy time with our grand management system which is the system the GMS system that you must use in order to apply. So that there are going to be plenty of requests, I think, for assistance and remind you of the help number for that GMS system which is 1888-549-9901. That number you will also find scattered through the program announcement and in the frequently asked questions available on the OJJDP web-site. Again, if we -- we don't get to your question today you can leave the message at 202-307-1341. I do want to remind you that messages left at that number after 05:00 pm Eastern time on Friday September 10th will not be returned. And you should not call this number that I have just given you for help with GMS. And I would want to also remind folks that after your application is submitted, no information about the status of the application is available until funding decisions are made in the spring of 2005. So I want to then turn to a couple of questions that have come in by e-mail. The first one came in from Michigan. Please clarify the difference between partner, applicant and co-applicant. As you recall, this is a collaborative request -- this program announcement requires a collaborative which involves at least two partners. One of those partners, the lead agency will be the applicant, the other partners are co-applicants. Any of the partners need to be evidenced by either a memorandum of understanding, a letter of agreement or some other indication of their commitment to this project. The second question, please clarify the submission deadline. Although the documentation says September 10th, there is also an indication that documentation as well as registration should be submitted by September 8th in order to allow you to review them for problems. That's not correct. The registration deadline is the September 8th deadline tomorrow and the final documentation needs to be submitted by the 10th. We are encouraging everyone to apply, to use the online system as soon as possible because we anticipate scores of applications being submitted and that there will be at the very least a queue and possibly, a long a wait in the queue in order to submit your application. At this point, I would like to see if there are any questions in general from -- from people about the program announcement itself. So Tania, can you open it up and tell people how to ask a question.

Tania: Yes. Ladies and gentlemen if you have a question at this time please press the "1" key on your touch-tone telephone. If your question has been answered or you wish to remove yourself from the queue, please press the "£" key. Our first question comes from Jane Sites. Your question please.

Jane Sites: Hi. As you can tell all of us are down to the wire and with those 30 pages are difficult. So after several of our folks have read your RFA, we still question two things. In those precious 30 pages, it seems to us that you are asking that the data source and collection information which we choose to present in a table format and the logic model are both in the text, meaning the 30 page count as well as you asked for it again in the attachment area. Is that your intention?

Bill Schechter: I think our intention and let me just check on that. Our intention was to have a description of the data collection and submission requirements in the program narrative. Any tables or the logic model should be in the single file other program attachment.

Jane Sites: Of the actual model.

Bill Schechter: Right.

Jane Sites: The graphs and boxes etc. --

Bill Schechter: Uh-huh.

Jane Sites: -- that's an attachment. And then you want a -- maybe a small description of that --

Bill Schechter: Right.

Jane Sites: -- in the text.

Bill Schechter: Right. And you should pay particular attention to the section in the program announcement that talks about how things are scored and make sure each of the sections reflects your response to those points.

Jane Sites: And our understanding of that is that other attachments might be labeled right as you have them bulleted in the RFA, correct?

Bill Schechter: Exactly, right.

Jane Sites: Okay. And then another little detailed question. This is again two and three are treated differently here. But as we understand it, the actual budget and the narrative are not a part of the 30 page count, is that correct?

Bill Schechter: That's correct.

Jane Sites: Okay. So they are in it -- they are outside of that number.

Bill Schechter: Uh-huh -- yes.

Jane Sites: All right. Thank you very much.

Female Speaker: Can we just clarify that, that what Jane was saying that the budget and budget narrative?

Jane Sites: Right.

Bill Schechter: Yes, right.

Female Speaker: Part of the 30 pages.

Bill Schechter: Thank you.

Tania: Our next question comes from Connie Black-Pond. Your question please.

Connie Black-Pond: Could you clarify whether indirect exposure by -- by definition includes the witnessing or knowledge of abuse to simply others living in the home besides adults?

Bill Schechter: Yes. It can.

Connie Black-Pond: Okay.

Bill Schechter: And that also -- you also submitted an e-mail question and I was about to get to that one as well.

Connie Black-Pond: Thank you.

Bill Schechter: Sure.

Tania: Our next question comes from Pat Schulte, your question please.

Pat Schulte: In the RSA, and talk about identifying the access within the partners that are supposed to work, could you clarify what you mean by that?

Bill Schechter: I think what that means is that you should describe the relationship of each of your partners to the project, what role they are going to play, what responsibility they have and how you will connect their various roles and responsibilities to the logic model and to the rational for the project you are proposing. Does that answer your question?

Pat Schulte: Yeah, thank you.

Tania: Our next question comes from Renee Aguilarie, your question please.

Kerry: Yes. Hi, actually this is Kerry from Anchorage Alaska. The question is related to the $10,000 set aside for data.

Bill Schechter: Could I ask you did perhaps just hold that for a few minutes, Katherine is going to talk about the data collection and -- and evaluation points in the program announcement in just a few minutes and if you could get this back in the queue and ask that question then.

Kerry: Okay. Thank you.

Bill Schechter: Thanks.

Tania: Our next question is from Dona Baldwin, your question please.

Dona Baldwin: Yes. On one of the pages that I read, it stated that 502-C3's are eligible, or 5013-C's are eligible.

Bill Schechter: I am not sure exactly where you read that, but if you are referring to non-profit organisations they usually have an IRS determination under a 501-C3 and yes they are definitely eligible.

Dona Baldwin: Thank you.

Tania: Our next question is from Tami Cardenas. Your question please.

Tami Cardenas: Hi, I had a question about the management section toward the end about the organizational chart. We are obviously, we are seeing different non-profit agencies who are collaborating, so in terms of the org chart would you be referring to it as a refresh to just the project or -- or charts for each of the individual agencies, how --

Bill Schechter: I think the most helpful thing would be to have an organizational chart for the project itself.

Tami Cardenas: Right.

Bill Schechter: And show how that relates to your partnership, your collaborative.

Tami Cardenas: Okay, that's what I was thinking. And my last--.

Bill Schechter: You -- you might want to include in other -- in the other attachment where it talks about supplemental material demonstrating organizational capability and readiness. You -- you may choose to include an organizational chart if particularly at the lead agency to show where administrative responsibility lies within that agency.

Tami Cardenas: Okay.

Bill Schechter: But that's optional.

Tami Cardenas: Okay, thank you. Am I allowed to ask another question?

Bill Schechter: Sure.

Tami Cardenas: I just had a question about the indirect cost in terms of the grant administration for the lead agency. Locally, usually we use between 10 to 12 percent. Is there are some sort of a guideline that you go by, I -- I looked in the FAQ's and it has this sheet about it but it's very unclear. It's just --

Bill Schechter: There -- there is no particular guideline. What's required is that whoever is asking for indirect cost must have either applied for or an actual letter of determination from your primary federal fund of a negotiated indirect cost, that's been audited and approved by that agency.

Tami Cardenas: Okay.

Bill Schechter: By that funding agency.

Tami Cardenas: Okay. Thank you.

Bill Schechter: So it might vary for the lead agency and say a subcontracting co-applicant.

Tami Cardenas: Do we have to submit that letters as part of our program?

Bill Schechter: That's right.

Tami Cardenas: So it's a letter from our federal --?

Bill Schechter: Right. It would be the letter of determination of your audited indirect cost that's been negotiated with them and it's usually the -- with the federal agency that provides the most federal funds to the agency and question.

Tami Cardenas: Okay, thank you.

Tania: Our next question comes from Shawn MacRae, your question please.

Shawn MacRae: Hi, I am just looking for clarification on the 30 pages. That's just based on the program narrative, right?

Bill Schechter: Yes.

Shawn MacRae: Okay, right, but and -- I had -- I did send an e-mail and maybe you will be getting to that, but I had a question on how to submit a Memorandum of Understanding with signatures via GMS?

Bill Schechter: That's a very good question and you are not the only person I have that e-mail actually right in front of me right now.

Shawn MacRae: Okay.

Bill Schechter: You -- you need to call the GMS help-desk at 888-549-9901 and tell them what you have and how to submit it and there is a provision, I think it's called right faxing where you fax it to them, they convert it to an electronic version that's added to your file in GMS.

Shawn MacRae: Okay, great.

Bill Schechter: And you did ask another question, which I mind as well answer at this point, if we budget in cash incentives for research participants they don't pay it out until the last interview at 24 months. Can we carry over those funds from year to year until year four? I think what you would do is you will be budgeting year-by-year and you would only budget the funds that you need in the year in question for that year. In other words don't count on the carry over.

Shawn MacRae: Okay, so -- so it -- it wouldn't be possible to hold over that payment until that fourth year?

Bill Schechter: Yes, it's -- it's a matter of how you define holding over.

Shawn MacRae: Okay.

Bill Schechter: Presumably if funds are available for all four years of funding and presuming that you have done a good job and better performance standards you would be submitting annual budgets and you would just calculate into each of those what you need to -- to cover those cash incentives for the follow on -- follow-up introducer.

Shawn MacRae: Okay. All right. I think I got that. Thank you very much.

Bill Schechter: Sure.

Tania: Our next question comes from Shelley Hamilton, your question please.

Shelley Hamilton: Hi, thank you for taking the call. I actually have two questions. The first actually relates to letters of commitment. We are submitting this application to as a part of a newly awarded Family Justice Centre grant that we have and a part of that grant, there is an MoU that goes along with all of the collaborative partners who will be a part of that. And I want to know could we submit that MoU as -- as part of the support documentation for this program here?

Bill Schechter: You could, but yes those same partners are going to be involved in the Safe Start project, then you would need to amend your MoU to reflect that and -- and have a signed amendment.

Shelley Hamilton: What we are going to do is submit the -- the large kind of Family Justice Centre MoU that shows all of these collaborative partners who are working under -- in the centre and then in addition to that have specific letters of support from key partners within that centre related specifically to the Safe Start program.

Bill Schechter: I suppose that would work as well. But the -- the thing I want to underscore is that what the reviewer is going to be looking for is evidence that you have a collaborative and that people's roles and responsibilities and commitment are clearly laid out and agreed to.

Shelley Hamilton: Okay, that's great and I have another question. For the budget, do we have to show all four years of the budget as a part of this initial proposal or we just do in the first year?

Bill Schechter: You could if you wish to, but you are really only need to submit the budget for the first year.

Shelley Hamilton: Great. Thank you.

Tania: Our next question is from Liz Dupont-Diehl. Your question please.

Liz Dupont-Diehl: Hi, thank you for taking my question. Regarding MoU's or signatures or other letter or document with signatures that we want to submit. If we have a capability here to scan them in and make a PDF file and sent that, is that all right or do you prefer us to fax it to the help desk? Bill Schechter: I believe that's okay. Yes, it is.

Liz Dupont-Diehl: Okay.

Bill Schechter: Because the -- I want to remind everybody that the other program attachments has called out on the application check list --

Liz Dupont-Diehl: Yes.

Bill Schechter: -- must include the following sections in a single file and that file can be in one of three formats a Microsoft Word doc format, a PDF file PDF format or a text document .txt. So it has to be in a single file.

Liz Dupont-Diehl: Okay.

Bill Schechter: So you will have to do some merging to make it a single file.

Liz Dupont-Diehl: Great.

Bill Schechter: But yes, that's perfectly acceptable.

Liz Dupont-Diehl: Also I am hearing that you like documentation of our indirect trade, would you like that to be included in the other program attachments?

Bill Schechter: Yes, that would be under supplemental material demonstrating organizational capacity -- capability readiness, would probably be the most appropriate place to put that.

Liz Dupont-Diehl: Okay. Is anything required under there? I see -- I see reference later on to an audit, but I am not clear whether that's required?

Bill Schechter: No. That I mean there will be -- depending on whether you meet the A1 33 requirements in your particular agency there will be audits the, the single audit required. But the threshold for that is now 500,000 a year and not everybody would meet that.

Female Speaker: Thank you.

Tania: Showing no further questions at this time.

Bill Schechter: Okay. At this time we want to move to Katherine who is going to deal with the data collection and evaluation aspects of the program announcement.

Katherine Darke-Schmitt: Good afternoon everyone. This is Katherine Darke-Schmitt from the office of Juvenile Justice and Delinquency Prevention and I am going to begin my section of the audio conference. I am reviewing some of the highlights of the data collection and evaluation corresponding to Safe Start Promising Approaches program and I need to begin by saying that there is no local evaluation requirement for site applicants you do not need to plan for or design a local evaluation. In fact we are asking folks who are considering including local evaluation plan to think very carefully about that because no plans that you plan to implement in the local level will be allowed to proceed if they impede or complicate in any way plans that will be underway by the national evaluator. The national evaluation will be awarded by contract or under a competitive process very similar to the one which site applicants are undergoing. Now we will happen to open the winter spring of 2005 and then that national evaluator who is selected through that competition will implement and design for the national evaluation in which all the local sites will participate. For that reason there was a $10,000 set aside for data collection only. The variables that are required to be collected are detailed in the backup program announcement. Applicants are responsible and successfully awarded, the applicants are responsible for collecting those data elements as many of them are as appropriate to the population that you serve for every child served. So the $10,000 can be used for any expense related to collecting the data, getting that data into a spreadsheet and sending the spreadsheet on to OJJDP. You will also see that there is a reference to comparison and control groups in the program announcement. This is because one of the agencies goals for this program is to produce outcome data on which we can make determinations about best practices. In order to be able to determine that something is the best practice, it must be evaluated with a comparison or a control design. However all that the applicants sites need to do in this respect is include in their applications where required a little bit of text describing how they would identify a comparison group or how they would implement random control -- random selection of a control group and I will revisit those two ideas in just a moment just want to make sure that you understand that the sites themselves this will have no responsibility for collecting data from the comparison or the control group. The national evaluator will budget for, pay for and do all the leg work associated with collecting information from the control and comparison group including the follow-up data which will occur at 12 months intervals at least twice during the length of the project. The program announcement makes it clear that we need for you to address two separate but related items, the first control group, second comparison group. Random assignment to a no treatment control group is sort of -- is the goal standard for program evaluation. We have a preference for picking up sites who are willing and able to be part of a random assignment group. This would mean that you would use methodology to randomly assign some children who are eligible to services to a group which would not receive services during the program period. Now many of you already have this condition in your sites because for instance you don't have enough funding or don't have enough staff to serve every single child in the jurisdiction who might qualify and so one approach to random assignment would be formalizing the selection of who receives the services and who doesn't to a random process rather than a where is their money to be able to put forward to deliver service process. We understand they are given the political and community context in some areas there will be applicants for whom random assignment to a no treatment control group is not politically feasible. If this is the case in your jurisdiction you must explain to us why that is and then go on to address the comparison group. Every application must comment with some description of a comparison group population which would be accessible to the national evaluator to collect data at the prescribed intervals. You need to give us ideas about where such a comparison group might come from, it is optional to provide any sort of letter of support or memoranda indicating agreement from that source, that is not a requirement of the application but it is one possible way you might document the fact that you had thought through this process. And then at the end of the selection process of the sites and at the end of the competition for the national evaluator, the national evaluator will work with the sites to determine which method is going to work best, comparison group or random assignment to control in each site and will work with the sites to finalize arrangements with the possible comparison or control groups. As I said the national evaluator has not been selected yet and as such I can't speak about the specifics of the design of the national evaluation, we don't have those in yet in application form. It is possible that a national evaluator may propose to use a local evaluator model in which -- under which structure they would contract with local evaluators at the sites to collect data on the comparison and control groups. If that were the case then the sites would again be involved in consultation with the national evaluator. So that we would understand what your concerns and needs are in identifying a local evaluator and the national evaluator and OJJDP would do it best to accommodate the sites, needs in terms of selecting a local evaluator. In just a few moments after taking some questions from you about evaluation, we are going to hear presentations by two organisations that were involved in the evaluation of Safe Start phase one, and so before you hear their presentations I want to make sure that you understand the context in which their remarks are being offered. Safe Start one was funded over 1998, 1999 it involved the selection of 11 original Safe Start sites which have been in operation since that time, Safe Start phase one program was tailored to systems change and collaboration building and as such the evaluation of those sites focuses heavily on the process of organizing systems change in collaboration building, however you will notice from reading the program announcement about Safe Start promising approaches which we refer to a Safe Start phase two that the focuses of phase one and phase two are quite different. Phase two is focused on service delivery and the collection and outcome data. Therefore when these evaluators talk to you about lessons learnt through the evaluation of phase one, we feel those lessons will be useful to you in thinking through your application and perhaps helping you identify what likely challenges and strategies for overcoming those challenges are likely to be, keep in mind they are not giving you direct advice about your application for phase two, since your focus will not be on collaboration building or systems change specifically but predominantly on the service delivery. And I will reiterate Bill's request that you not wait until the last minute to make your application, this system does turn off at 8:00 pm eastern time on Friday, September 10th. We cannot turn it back on. So no matter what happens between now and then you need to have your applications in. If you need assistance with the grants management system please call that 888 number that Bill read to you and which is on page lower case III of the application. Although the program staff can address programmatic questions we are not technical analysts and we cannot answer questions about the GMS systems, and it is above and beyond us. At this point I am going to turn to Tania if you would open the phone I will address 10 minutes of questions about data collection and evaluation. And then we will precede the remarks by the Safe Start phase one evaluators and the you will have another opportunity to ask questions about their remarks.

Tania: Yes, if you have a question at this time please press the "1" key on your touch-tone telephone. Our first question comes from Renee Aguilarie, your question please.

Kerry: Yes, good morning actually it's Kerry, and the question from Anchorage Alaska is related to the $210,000 of which 10,000 is budgeted for the data collection, is it possible to use up to 20,000 for the data collection?

Female Speaker: It is possible the re-viewers would not automatically screen out your application however if you did that your application might look significantly different from the applications submitted by other agencies and sometimes difference is not a good thing.

Female Speaker: Okay -- okay, thank you.

Female Speaker: Sure.

Tania: Our next question comes from Pat Schulte your question please.

Pat Schulte: Hi, I just wanted a clarification regarding the control group and the comparison groups. If we determined that probably a comparison group would work better at our sites we still need to have a section stating why we don't think our control group will work.

Female Speaker: That is right you can't just ignore the question you need to tell us in a couple of sentences specifically why a control group is not possible.

Pat Schulte: Okay, could you define the difference between control and comparison again, I know you have addressed it but if you --.

Female Speaker: I am going to refer you to the sources identified in the program announcement and to the web site which will give you some technical background on the differences between those two.

Pat Schulte: Okay, thank you.

Tania: Our next question is from Tammy Cardenas, your question please.

Tammy Cardenas: Hello, I have a question about the incentives so in the RFA that takes about the comparison and control group plans. And one of the related questions to the control group is how we are going to retain -- retain them for the -- follow up data collection if -- if we want to include incentives for our participating clients in our budget do we also have to include incentives for the control groups in the budget or not?

Female Speaker: You do not, if incentives are to be offered to your control group participants and it's entirely possible that they would be, that would come out of the national evaluators budget, not yours.

Tammy Cardenas: So we can just suggest that in this section like we have suggested an incentive, but we don't have to pay for it.

Female Speaker: Correct.

Tammy Cardenas: Okay, thank you.

Female Speaker: You are welcome.

Tania: And if you have a question at this time please press the "1" key. Our next question is from Sherry Davis.

Sherry Davis: Yes, can you talk a little bit about whether there is a relationship between the IRB and the evaluation, itself that we are speaking of now?

Female Speaker: The relationship between the IRB is that when one uses federal dollars to conduct research on human subjects in ways where data would be collected on individuals and institutional review board which we call an IRB in short hand, must have given you approval that your plans are sound, that the gains to be earned by the research out weigh any risk or harm to the participants and that you have policies and practices in place which will best protect the safety and the privacy of your participants. So those two ideas are conjunctionally related as parts of the evaluation.

Sherry Davis: Okay, and if we don't have presently an IRB relationship we should follow your directions and just -- include dollars in the proposal for that to occur?

Female Speaker: That's correct.

Sherry Davis: Okay, thank you.

Female Speaker: You are welcome.

Tania: Showing no further questions at this time.

Female Speaker: Thank you Tania, at this point then we are going to hear from the first of two organisations who are involved in the evaluation of Safe Start phase one. This first organization is Caliber Associates of Surfex, Virginia, go ahead Susan or Lisa when you are ready.

Susan Chibnall: Hi, this is Susan Chibnall, before I just make several comments to you I just wanted to reiterate what Katherine was saying, which is -- that these most of these issues are things that you might want to consider, just thinking about as you finalize your plans and they are actually from the Phase one, Safe Start but I don't think there are -- there are any less salient to you even the -- the new focus of the new Safe Start initiative. I am -- I am actually going to just -- just going to talk about three areas, the first one is just to talk to you a little bit about new things we learnt from the collaborative, the second is just a couple of general statements I am going to make and then the final are a few statements about implementation specifically that is implementing your interventions that are designed to specifically for this -- children and families that you will be working with. Some of this may not be new to you, and for some of you it may be things that you already know and know very well. But I think that with our experience with this initiatives it is just important to reiterate them so that you are thinking about them in the forefront of your mind as you are finalizing your applications. So the first piece I am going to talk just a little bit about is your collaborative, and one of the things that we found in the process of evaluation was that the structure and organization of your collaborative body can really have an impact on your work, and so there is a couple of issues that you might want to think about such as who has represented your cable and why, are the right people there, and even more important who are the right people. What is the best structure for getting your work accomplished in the phase one Safe Start initiative, they were a variety of different collaborative structures including small management bodies within a larger group -- smaller working groups that had specific activities and goals to accomplish. And the manner by which the collaborative was structured was really important to how the work was completed. Its also important to think about your decision making process -- specifically who will be involved in that process and how will decisions be made and ultimately -- who gets the final say in decisions. And that sort of moves into its the nice segment into my final point -- about the structure and organization which is how will you manage several systems with separate agendas simultaneously. Obviously it could be a tricky for a variety of reasons but it's something to think about -- at the out set of your program. The second point, I want to make -- and final point about the collaborative is getting everyone on the same page. Mean while you may even think that your partners are the partners that you are planning to work with on this initiative are already aware of the issues around children exposed to violence. People may have different understandings or different perceptions of what the issues are and so it's really important to get everyone on the same page early on. And so if there is any training among service providers or collaborative representatives that is going to help bring people -- to the same point with relation to the issues and also in particular your intervention its, important to do that very early on. We found that having people on the same page doing training and -- those kinds of things early on really increases awareness of the -- important issues and also -- helps to define what the important issues are in your community and also really increases commitment to the process. I am going to leave the collaborative piece behind now and just -- just say a few general statements -- which is first of all to never underestimate the value of time I mean this -- is really a process and it needs to have some time to unfold -- in particular I think that we are sometimes more optimistic -- about how long things will take or how -- or how much time things will take. So it's important to think of things that through. Another point which I think goes along with the importance of being realistic about time is that you should also think about setting resources aside for the unexpected -- while you certainly can't anticipate everything -- probably as a experienced service providers and administrators there are something's you can pull from your experiences that might help you to anticipate some challenges or some barriers or even just something's that may -- take your plans slightly off-line. So it's just important that you have the resources you are going to need for the unexpected. And then finally it's -- it's really important especially in these large initiatives that -- perhaps many of you had experience with them to really stay focused in scope, particularly when the collaborative body involved -- there is a lot of agendas a lot of perspectives, a lot of philosophy present at the table -- but we have found that those collaborative that we are really able to stay focused on the issues on what they were doing were better able to be successful. And then finally I am just going to -- add a couple of things to think about in with respect to implementation and that is implementation specifically around your intervention. So when you are thinking about -- implementing your intervention its really important to think about what barriers you might expect to experience -- in identifying, assessing, referring and treating your intervention kids and families. I think its really important to anticipate these early I think that -- initial phase of Safe Start phase one there was just sort of -- some thinking that if its available people will come to it and that wasn't always the case and so its important again to think about how you are going to find and assess the kids, how you are going to refer with them, and then how are you going to get them and maintain them in treatment. You also need to think about these -- about your community in relations to these potential barriers and facilitators mean for example world -- communities may experience barriers different from those experienced by urban communities, so if you are a world community you may have issues with transportation, if you are a urban community that has a world population -- you may need to think about barriers that may occur for those folks that are different from those living in mere more urban areas. It's also important to try to anticipate challenges to finding qualified personnel to match your program component. If you are implementing new intervention and have to hire new staff for those interventions, it's important that you not under estimate the amount of time it may take you to find truly qualified individuals to implement your intervention. And then finally, it's really important to examine where your strategy that is the interventions you are going to implement, really fit in the context of each organization involved, and how differing agendas, priorities and philosophies may affect implementation. And I think that's another, sometimes we may think that because we are all service providers and here to help children that our agendas don't get in the way of providing good services to children. But when you have a lot of players involved it's really important to understand how those agendas and priorities and philosophies may sentence, -- may affect the extend to which you can, really effectively implement your intervention. So that's all I have to say. I hope you find those comments helpful. Anybody have anything to add?

Female speaker: Okay, thanks Katherine.

Katherine Darke-Schmitt: Thank you very much Caliber. The next speaker comes from the Association for Study and Development of -- of the Community. They are the second partner in the national evaluation of Safe Start phase one. This is Inga James.

Inga James: Hello everybody. From the beginning of Safe Start phase one ASTC, my company had provided technical assistance to the local evaluation teams and some of the things I would like to discuss with you today, have to do with developing internal capacity for data collection and some of the things that our sites have found that they need to improve or enhance and order to, collect the data that they needed to collect and then manage it, as they were going along. And again I am going to present you with some things that have challenged, besides that we are currently working with, and ways in which they have resolved some of these challenges. With the understanding that you may or may not ever run into these challenges, they may not be something the is problematic for you. So it's kind of a lessons learnt, idea here. The first thing that I wanted to talk about, how to do with developing internal capacity to collect data. First of all, and this sounds very much like common sense, but I -- but some of the sites that we have worked with, did not think about this in advance, and so I'm passing this wisdom on to you and that is that, the data collection method and systems, that are set up to collect data for Safe Start phase two, should exist within your -- with your already, within your already existing data collection systems. So trying to superimpose something new on your existing system, is probably going to not work out too well, especially given the time frame that you are being asked to provide these services then. And - and it's it also works best, if the data collection systems build on the existing systems that you have in your program, and this includes any kind of information collecting systems that you have. And one of the things that we recommend is that sites used off the shelf data base systems or software, because trying to tailor something to Safe Start, basically probably means that, unless you are already using that system, it will be brand new to your program, and may create problems. A second -- second idea that I wanted to share with you about developing internal capacity, has to do with receiving consent from the families to participate in research. Our sites found that families had no, generally had no problems signing consent to participate in the program but that they sometimes were hesitant to sign consent to participate in the research, for whatever reason and that they had to work quite hard to help their families understand what the research was about, where the information would go, and what would be done with it. This seemed to be a particular problem for immigrant families, primarily because of not knowing exactly what information will be passed on to whom. Some tips that -- that we found or that we learnt along the way, have to do with making sure that consents come in several languages, and that's common sense as well, and that some of our sites worked with other community agencies who also had relationships with these families to -- to help the families understand what was, what the research was all about and what the consent was all about. The last thing about developing internal capacity is that line staff need to be brought into the data collection process and that, that steps should be taken that they understand the value of data collection and data management and that is -- the concept of data collection is translated into something that's meaningful for them, whether it's about case management on their part or whatever it might be that they understand that data collection is all part of the bigger picture. The second area I wanted to talk to you about has to do with preparing for assessment in data collection and management of data. Sites that we worked were sound -- it quite a task to either develop or utilize any of the existing assessment tools to assess children who have been exposed to violence. They found that they spend a lot of time choosing the right assessment tool, training people to perform conductive assessment and then working with interpreting the assessments. So, I think, the more upfront time you can put into developing assessments and training staff to use the assessments, you are going to find that, that's very helpful to the process. Things that need -- you need to keep in mind or at least we found we needed to keep in mind had to do with keeping the length of time for assessment down to a minimum. We found that both clinicians and families tended to get quite vary of some of the assessment processes that were used by sites that the -- the assessment should be age and culturally appropriate and that the skills of the assessor need to be enhanced given -- depending on the assessment tool that's used. Often times communities will have local academic or other expertise available to help them select the appropriate assessment tool for their program. We have on the "capacitybuilding.net" web-site that Bill mentioned earlier a list of measures that has been -- some of it have been used, some have been tried and discarded, but there is a list of measures along with their profiles on that web-site, if you wanted to take a look at that and see what would work with your population and your community. As far as other tips for preparing for data collection and management, it's important to know what your contractors -- if you will be working with contract staff such as mental health clinicians, or case managers or early childhood specialists, understand what their capacities are regarding data collection and -- and know where they may need training or capacity building and that again is something that's best done upfront as opposed to finding out six months down the road that maybe the capacity is lacking. And this goes as well for partner agencies and understanding if they are going to be involved in any kind of case management or data collection, what their capacities are and what their training needs might be. One way that, some of the sites found and that it may be too late in the game for me to even mention this, but some of the sites found, it was helpful to involve the partner agencies in actual composition of the RFP or the proposal, because then it became more clear during discussions that just exactly what partner agencies capacities were and what might need to be enhanced. And I think, somebody already mentioned the IRB approval, and that process and I am happy to discuss that more, if there are further questions, I know there is also some information on the frequently asked questions and web-sites. So, it just -- know that it's important that you have IRB approval and that, that approvals can sometimes take a significant amount of time, or much more time than is expected, depending on how your IRB, your board works, and how they goes for the approval process. One specific tip I wanted to share with you has to do with how to write up the IRB proposal. Many of the sites submitted proposals to their internal review board asking only for approval to submit aggregate data to OJJDP and the national evaluation team. What it found what you will need to do is to ask for approval to send case level data and that's very different from sending aggregate data and the IRB will -- will look at it differently knowing that you are sending case level data. So it's important that you make that distinction in your proposal. The last you know, I would like to talk about has to do with developing comparison groups or controls groups for your site again this is -- it's not your responsibility to develop the comparison or control groups but the national evaluation team is more than likely going to look to you to help set those up since you will know your communities and know where they can come from. And just a few tips to keep in mind one is that although it's not an OJJDP requirement it is a scientific standard that sites recruit at least 30 members for each comparison or control group and 30 is the minimum numbers so with a trichion it should be greater than 30 and so when your -- when your writing at a comparison group plan keep in mind that what ever organization or how ever you go about developing your comparison group plan you should be able to attract 30 different people. One idea in doing this aside from the idea that was already mentioned by Katherine about developing a control group from a weight listed of group of kids it is to work with other agencies who may be serving a similar population but to whom you don't expect to provide services. So -- so these other agencies will be serving children who have been exposed to violence but either because of funding or because of location or because of any number of reasons these children will not be served by Safe Start that can provide an adequate comparison group some sites for example in phase one are working with Hedge star sites and are not providing these Hedge star children with services but these children have been assessed to show that they have been expose to violence. You can also looked un-served neighborhoods -- neighborhoods in which your funding won't reach that may have a similar demographics and similar population. And the other -- the last thing is that although working with comparison or control group children will not be your responsibility there will need to be a plan put in place for retaining these children over the course of the life of the study and the national evaluation team, again we will probably be looking to the individual sites to help them develop that plan, attrition in comparison and control groups tends to be rather high if attention is not put to retaining those individuals and so that will be a priority for the national evaluation team to develop a plan to maintain those children in the research study and that's all I have for you today and I will turn it back over to Katherine.

Katherine Darke-Schmitt: Thank you so much for your remarks Anga and again thank your Caliber, Tania we are ready now to open the line to questions again.

Tania: Ladies and gentlemen if you have a question at this time please press the "1" key. Our first question is from Sherry Davis your question please.

Sherry Davis: Yes, my question is regarding eligibility. There was one clause in there because we have several types of community partners and one question was -- who is eligible and something that for profit except for small businesses. Can you help us interpret that? Female Speaker: Yes we are looking through our --

Bill Schechter: I don't recall where it says that.

Female Speaker: -- on the --

Bill Schechter: May be that's not correct.

Female Speaker: -- that's not correct.

Bill Schechter: No -- no.

Female Speaker: Because we say it on one of the Q&A I believe it was. Can you help me with the correct definition.

Bill Schechter: Yeah.

Female speaker: Yeah. It's on page 18 of the -- of the program announcement under the paragraph labeled eligibility.

Female Speaker: So you are saying that where I read that --that's not correct and the correct statement is what.

Female speaker: What I am saying is the correct statement is that any private organization that agrees to raise profit or fee it's eligible and that if you read a transcript about discussing that issue on a previous audio conference you may have been reading the question about whether or not a small business was eligible but small business have always been eligible as long as they are not collecting a profit or a fee.

Female speaker: Okay. And so that means that they would put the statement in the application as one of the partners.

Female Speaker: That's right.

Female Speaker: Okay, thank you so much.

Female Speaker: Sure.

Tania: Again if you have a question at this time please press the "1" key. I am not showing any questions at this time.

Female Speaker: Okay. Can we open that up for another 30 seconds, Tania?

Tania: Sure.

Female speaker: Well, the line is open for folks to ask additional question there, it one have come in Via e-mail the question is the Safe Start grand requires control group, does that mean that organisations providing other services to the same population group must exclude clients previously identified as needing an intervention, no it does not. Second the level of research data management and so on is substantial at the level only universities have the infrastructure to provide our university is the primary applicants for this grand again, no there is no research requirement at all under this grand and the only data management that is required is loading the variables collected quarterly and twice yearly in to a spread sheet so we feel that's entirely manageable by community agencies. You are welcomed to have the university as a partner or even the primary applicant on the grant if that works for your community, but we don't feel like their subsequent expertise is particularly necessary because we have designed this program so that the national evaluator can come in and implement the research and evaluation strategy. The last part of this question was can most world or tribal non-profit organisations provide the necessary data collection in management, and yes we feel that they can. Tania do we have questions in the queue.

Tania: We do have a question from Michael Lane your question please. There are no further question.

Female Speaker: Okay. Bill is there anything further we would need to cover in this call.

Bill Schechter: I don't think there is.

AMBER Alert | Faith-Based & Community Initiatives | Helping America's Youth
National Sex Offender Public Registry | OJP Reentry Initiative

USA.gov | Privacy | Policies & Disclaimers | FOIA | Site Map | AskJJ
A component of the Office of Justice Programs, U.S. Department of Justice