The Downtown Steering Committee is conducting a survey, with a wide variety of questions. “06880” reader Jeff Block thinks there are problems. He writes:
The Downtown Steering Committee (DSC), under the guidance of urban planning consultant RBA, has made available a public survey intended to glean from Westport residents a vision of what the town could look like in the future.
The intention is sound. However, the survey and the data-gathering process are flawed. To date there have been 500+ responses to the survey, which is available both electronically and in hard copy. The survey specifically targets responses from Westport residents, but there are no controls or requirements that can be used to identify that the person filling out the survey is in fact from Westport. Additionally, the DSC cannot prevent individuals from inputting more than one response to the survey.
Downtown Westport: the subject of planning, and a survey.
During the initial issuance of the electronic survey, committee members realized that people could send the survey in multiple times. One DSC member actually noted that he had submitted it on at least 2 occasions.
The DSC revisited their process and built in a control using the computer device’s IP address, to prevent anyone from answering the survey more than once from the same device. However, anyone with more than one device, for example an iPad, mobile phone or additional computer, could submit as many surveys as devices available.
Since there is no requirement to identify respondents, it is impossible to know how many times an individual may have submitted multiple surveys, and more importantly if the person responding is even a Westport resident.
The data gathered to date cannot be relied on. In view of the facts, the DSC needs to respect its goal of public transparency, bite the bullet, revisit their processes and reissue the survey, incorporating a tested set of well designed controls.
Downtown Westport comprises a very small section of town. But its impact — economic, aesthetic and psychic — is huge.
Dewey Loselle and Melissa Kane — general chair and public outreach chair of the committee — respond:
It is disappointing that there are a number of erroneous facts, misinformation and misguided conclusions circulating that give a false impression about the validity and utility of the survey.
1) More than 750 completed on-line surveys have been received in roughly 2 weeks. This is a very high response rate in such a short time. We have reached out to all kinds of community organizations — over 50 in total — and asked them to urge their memberships to participate in the survey. We have already received a very high level of cooperation and support from many of the organizations. Clearly, Westporters are very interested and are engaging with the process.
2) The survey is meant to capture general sentiments, values, and the ideas of Westport residents regarding the future of downtown. It is also meant to get Westport residents to think about downtown, their relationship with it and their vision for the future of downtown, and encourage them to be engaged in the planning process.
The survey does not represent a vote or a referendum on any downtown issues. The responses to questions are purposely structured to avoid the use of “yes/no” questions. The responses provide a spectrum of choices (e.g., “very important,” “somewhat important,” “I don’t know,”) that reflect individual priorities and preferences among an array of issues.
A gem, hidden in plain sight downtown. Survey questions ask about Westport’s use of the riverfront.
3) No one can send in a response more than once from the same device. A safeguard permits only 1 response per IP address. Early on we allowed testers (and a few early responders) to change their responses to a previously submitted survey. This, however, still only allowed for 1 survey to be counted.
While it is possible that someone could submit more than 1 response from a different device, we believe this to be unlikely. It is difficult enough to get people to respond to the survey once.
Similarly, we believe the concern that many out-of-towners will respond to be a non-issue. What would be the motivation? There are opportunities for non-Westporters to comment on the process on our website, as well as through a separate Downtown Merchants Association survey. The law of large numbers posits that valid survey respondents will overwhelm any few potential miscreants who enter an additional survey from another device, as well as anyone from out of town who might decide to respond.
Should Main Street be a pedestrian mall?
4) Every survey involves tradeoffs depending on the topic, level of security needed, cost and participation rate desired. We considered asking people to provide their first and last names and address, but felt that might discourage many from taking the survey. We decided to ask (optionally) for the name of the street on which a respondent lives. We balanced the idea of capturing personal information with the need to get people engaged, encouraged to respond and motivated about this planning process.
5) We have the ability to review and compare individual survey entries. We will flag open-ended responses that appear to be unusually similar, or that have conflicting responses, and evaluate them more closely to determine if any of them should be rejected. Results of the survey will be presented in a public forum and published on our website along with other relevant studies, project deliverables and surveys.
We believe this fully explains the transparency of this survey, and the common sense processes and controls. There is no reason to reissue the survey.
We urge every resident who has not yet taken the survey to do so. The Your Downtown survey is available at www.westportdowntownct.com.
Why wouldn’t you want people (shoppers) from outside Westport to chime in? They make up a huge percentage of shoppers. #deathbycommittee
Law of large numbers? I think a fairly large number of people read this blog. And, judging by the comments, a substantial percentage do not currently live in Westport but have strong opinions about how downtown should look. These individuals are also connected to the internet and have a surprising amount of time on their hands. They have also shown a love for offering anonymous opinions about Westport that has been curtailed for a little over a year. These factors might lead to a fairly large number of opinions coming in that sway the impression that the committee gets. 750 in the last 2 weeks? How many come in today from the Kingmaker?
Thank you to Dewey and Melissa. We realize that we can never satisfy everyone. Keep up the good work.
First I want to go on record that I support the DSC goals of bringing a semblance of predictability to how Westport May grow in the future. While I appreciate Dewey’s well thought out reply and explanation it still did not properly address controls. The law of averages approach that Dewey has identified does not address the ways that cyberspace can be compromised. The direction of public opinion that will be interpreted from survey replies should not be assumed to be statistically valid given the number of replies and the fact that using IP addresses is a weak control.
You cannot assume that a publically available survey, capable of being addressed by anyone accessing it over the internet, could not be compromised.
DSC clearly will not and likely cannot reissue the survey at this time but they need to be more cognizant of their approach to similar tasks they will be undertaking as the process moves forward.
By design, voluntary web surveys are statistically meaningless. While there may be some anecdotal value gleaned from data derived from a highly motivated, non-randomized group of respondents, to suggest that the results would be representative of the Westport electorate’s sentiment is incorrect.
When the survey is published, I suggest disregarding the fancy charts and graphs used to present the flawed quantitative data and focus on the qualitative verbatims (what people actually say on the open-ended questions). Hopefully those won’t get flagged and deleted as mentioned in item #5 above.
In response to Mr. Blocks letter as a Certified Planner, a partner in the RBA Group, the firm entrusted by the Town to conduct the Master Plan study and most importantly a resident of Westport we took great care in developing the survey to generate public opinion while also respecting private citizens anonymity. Therefore let me offer the following
RBA , as normal practice does not ask for personal information in a community survey unless it is an optional item (e.g., “please provide your email if you would like to receive updates,” etc.) We decided to ask for the name of the street on which a respondent lives (and made this optional), in addition to questions about age, etc. Again, most surveys make providing demographic information optional. We balanced the idea of capturing personal information with the need to get people engaged, encouraged to respond and motivated about this planning process. And again, this is a survey – a sharing of ideas and aspirations – not a vote or a referendum.
In addition a more secure survey (although there is no 100% secure survey) would involve creating a mechanism whereby every adult resident of Westport was provided a unique code to take an online survey. He or she would be mailed the code (there are more than 9,000 households in Westport) and given the opportunity to use the code to fill out the survey online. This method is typically used for elections and voting in communities. We are not electing anyone nor are we voting on any issues. This method costs much more, would require greater time to administer and receive feedback, would likely result in far fewer responses
and is uncalled for in this type of survey.
Most importantly the results of the survey will not be transferred directly into the Downtown Westport Master Plan as Town policy. Furthermore, the survey is not the sole public engagement opportunity as part of this Downtown Westport Master Plan process. The Downtown Visioning Workshop and two-day Charrette are in-person events for residents to participate in a more active manner.
David Lapping, PTP
Senior Vice President, Director
New York | Connecticut | Philadelphia
dlapping@rbagroup.com
How much did this survey cost the town? Who is going to weed out wonky returns? (Can I have that job? Reasonable fee…)
Two quick points.
First, I moved away from Westport in 2008, and seriously considered taking this survey. I clicked the link but then the phone rang and “squirrel!” – I never came back to it.
Second, if the survey results can’t be “transferred directly” what is the point?
I hope somewhere along this process, the townspeople, the Westport residents, will get a chance to vote on what is being put forward to have done down town. And no one answered the question about what this survey is costing the taxpayers, or for that matter, what this company, RBA Group, is costing the taxpayers. And I’d like to know how this company was chosen? Who picked it, was there a competitive bid, who initiated the contact?
And putting downtown aside, what is happening to the future plans for the rest of Westport’s business sector? The Post Rd to Fairfield, and the Post Rd to Norwalk?
The Downtown Steering Committee, Chaired by Dewey Loselle, and RBA are doing a pretty darn good job in reaching out and trying to get engagement from citizens. In my view, the engagement aspect is as important as the substantive comments. It is easy to criticize and hard to make everyone happy. RBA is going about its work in a professional manner. Let’s all do our best to cause whatever change may occur downtown to make sense to and benefit most Westporters. Don Bergmann