[Student-projects] Regarding GSoc 2014

Tue Mar 11 01:39:18 PDT 2014

Deepak,

It doesn't work like that. We cannot parse MARC21 data with fixed length
strings. Here's the MARC21 tutorial -
http://www.loc.gov/marc/umb/um01to06.html

On Tue, Mar 11, 2014 at 2:04 PM, Deepak kumar <
deepak.kumar.ece11 at itbhu.ac.in> wrote:

> In view
> <label for="file">Upload text File</label> <%= file_field_tag "file" %>
> In controller
> uploaded_file = params[:file]
> @file_content = uploaded_file.read
>
> In this way @file_content becomes an array of whole record.
> and by indexing I extracted fields like for leader
> @leader=@file_content[0..23] since the leader is 24 char. long.
>
>
>
>
> On Tue, Mar 11, 2014 at 1:59 PM, Ershad K <ershad92 at gmail.com> wrote:
>
>> Hi Deepak,
>>
>> Can you explain how you parsed the MARC21 file? Also always use proper
>> classes/controllers, it's good to move such logic out of
>> UsersController#new.
>>
>>
>> On Tue, Mar 11, 2014 at 1:34 PM, Deepak kumar <
>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>
>>> the link of code https://github.com/deepakrocks0009/trial
>>>
>>>
>>> On Tue, Mar 11, 2014 at 1:34 PM, Deepak kumar <
>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>
>>>> Hello Sir
>>>>
>>>> I have taken example from http://www.loc.gov/marc/umb/um11to12.html
>>>> I tried to read MARC record from a text file and output tag, length,
>>>> start_at and data.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Mar 11, 2014 at 1:57 AM, Ershad K <ershad92 at gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 11, 2014 at 1:53 AM, Deepak kumar <
>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>
>>>>>> https://github.com/deepakrocks0009/blog  But sir please don't judge
>>>>>> me on the basis of this code as I was just beginner at that time.
>>>>>>
>>>>>
>>>>> No worries about that, everybody is beginner at sometime. I just
>>>>> wanted to see some code in action.
>>>>>
>>>>>
>>>>>>
>>>>>> So do I need to create database for this or just reading the record
>>>>>> from a file?
>>>>>>
>>>>>
>>>>> It's your choice, but for now just parsing a valid MARC21 record
>>>>> (which you need to find out) is completely fine.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 11, 2014 at 1:48 AM, Ershad K <ershad92 at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Deepak,
>>>>>>>
>>>>>>> Please see if you find and parse a MARC21 record. You are free to
>>>>>>> use any gem.
>>>>>>>
>>>>>>> Is deepakkeshri.herokuapp.com open source? Can I see its source
>>>>>>> please?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 11, 2014 at 1:42 AM, Deepak kumar <
>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>
>>>>>>>> This was my blog app http://deepakkeshri.herokuapp.com/ which I
>>>>>>>> made when I was just a beginner. After that I learnt a lot of things.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Mar 11, 2014 at 1:34 AM, Deepak kumar <
>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>
>>>>>>>>> Hello Bhaiya,
>>>>>>>>>
>>>>>>>>> You are right. Using STI we will reduce the number of tables.
>>>>>>>>>
>>>>>>>>> My github link is github.com/deepakrocks0009. But I dont have any
>>>>>>>>> big project in my account. The project on which I worked during 2013 summer
>>>>>>>>> was schoolmitra.com and since it was private company they closed
>>>>>>>>> my bitbucket account after internship.
>>>>>>>>> Till now I haven't tried to parse MARC records. But you can give
>>>>>>>>> me task before I submit my proposal.
>>>>>>>>>
>>>>>>>>> Ya please put this thread to mailing list.
>>>>>>>>> How many applications do you received for this project?
>>>>>>>>> Kind Regards
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Mar 11, 2014 at 12:55 AM, Ershad K <ershad92 at gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> Hi Deepak,
>>>>>>>>>>
>>>>>>>>>> Yeah, but we could make it better using rails's STI. Also have
>>>>>>>>>> you tried to parse the MARC records? Can you please link to the projects
>>>>>>>>>> you worked on? Preferably in rails?
>>>>>>>>>>
>>>>>>>>>> Can I CC this thread to student-projects at lists.smc.org.in ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> PS: Don't call me sir, call me Ershad :-)
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 10, 2014 at 10:54 PM, Deepak kumar <
>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Sir,
>>>>>>>>>>> We can have two tables to store MARC 21 data. First table will
>>>>>>>>>>> contain columns which are field_id, field tag, its value. 2nd table will
>>>>>>>>>>> contain sub field tag and its value differentiated on the basis of
>>>>>>>>>>> field_id.
>>>>>>>>>>> We need to have an extra default row in the table. If the field
>>>>>>>>>>> doesn't matches existing fields of db then the default row will be used.
>>>>>>>>>>> Is this correct?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 10, 2014 at 10:17 PM, Ershad K <ershad92 at gmail.com>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>
>>>>>>>>>>>> Do you have any idea about how to store these data in database?
>>>>>>>>>>>> What happens when we encounter new data fields while importing the data?
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Mar 10, 2014 at 6:30 PM, Deepak kumar <
>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> You are right sir, the database for the fields is really huge.
>>>>>>>>>>>>> It has 887 control fields and there sub fields, so total of 6000+ field
>>>>>>>>>>>>> types.
>>>>>>>>>>>>> It is very difficult to have a look up table for each field.
>>>>>>>>>>>>> We have to use very good data structure. but there are large number of
>>>>>>>>>>>>> tools for decoding. We can take help of them.
>>>>>>>>>>>>> http://www.loc.gov/marc/marctools.html
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Mar 9, 2014 at 3:20 PM, Deepak kumar <
>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you sir for your reply.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am quite good with ruby on rails and its mvc architecture.
>>>>>>>>>>>>>> The project on which I had worked was www.schoolmitra.com.
>>>>>>>>>>>>>> It was basically web application using ruby and its framework RoR.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have gone through your blog and your weekly update during
>>>>>>>>>>>>>> GSOC'13 and it is quite impressive.
>>>>>>>>>>>>>> Some Similar works I also did during my summer internship.
>>>>>>>>>>>>>> Implementing authentication, post comment and like feature, student
>>>>>>>>>>>>>> database management, monthly report generation in PDFs form, defining role
>>>>>>>>>>>>>> through rolify and ability through cancan, bootstrap for css, SQlite
>>>>>>>>>>>>>> formanaging database, sms alert functionality, javaScript and little jquery
>>>>>>>>>>>>>> for dynamic pages. But I don't know haml. I used to make my view page in
>>>>>>>>>>>>>> ruby only. I will learn it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kind Regards
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Mar 8, 2014 at 1:31 PM, Ershad K <ershad92 at gmail.com>wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sorry, I was busy, wasn't able to send you a reply.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Mar 6, 2014 at 1:34 PM, Deepak kumar <
>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello Sir,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As suggested, I have gone through the documentation of
>>>>>>>>>>>>>>>> MARC21. What I have understood is, MARC21 is a basically formatting of
>>>>>>>>>>>>>>>> certain records/data either bibliographic, authority, holdings, or
>>>>>>>>>>>>>>>> community information. In other words we can say it as encoding and
>>>>>>>>>>>>>>>> decoding process of records in a fixed format. Today MARC21 is most
>>>>>>>>>>>>>>>> predominant and widely used. So in order to import data from a MARC21
>>>>>>>>>>>>>>>> record we have to read the record character by character and perform
>>>>>>>>>>>>>>>> decoding operation. The project says we have to import and export data in
>>>>>>>>>>>>>>>> MARC21 records. The only tough thing about this project is to know the
>>>>>>>>>>>>>>>> MARC21 format.
>>>>>>>>>>>>>>>> So my steps will be
>>>>>>>>>>>>>>>> 1. Reading the MARC21 record file as a string or reading
>>>>>>>>>>>>>>>> fixed number of character from record each time instead of reading whole
>>>>>>>>>>>>>>>> record as single string.
>>>>>>>>>>>>>>>> 2. Assigning these fixed length strings to fixed
>>>>>>>>>>>>>>>> variables.(Tag,length,start position,end position)
>>>>>>>>>>>>>>>> 3. With the help of these tags reading the rest of file.
>>>>>>>>>>>>>>>> 4. Lastly formatting these pieces of data in fixed format
>>>>>>>>>>>>>>>> for end users with the help of inbuilt database.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There's some problem here. First, we cannot really read
>>>>>>>>>>>>>>> MARC21 records by reading with a fixed number of characters. Last time I
>>>>>>>>>>>>>>> checked, it has around 3000+ different fields with different type of data.
>>>>>>>>>>>>>>> Each file has many subfield marked with leader tags or something. We cannot
>>>>>>>>>>>>>>> proceed with this project until we have a clear understanding of how MARC21
>>>>>>>>>>>>>>> works.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In addition to that, the scope of this project is not merely
>>>>>>>>>>>>>>> parsing the MARC21 data. We have to make necessary changes in Grandham
>>>>>>>>>>>>>>> application to add import/export feature. That includes changes in the
>>>>>>>>>>>>>>> current DB schema. How good are you with Ruby on Rails?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This will be basic idea of my work. Please correct me
>>>>>>>>>>>>>>>> wherever I am wrong also if I am missing any point.
>>>>>>>>>>>>>>>> Please tell when you are available on IRC . I have posted
>>>>>>>>>>>>>>>> this mail to mailing list but didn't got any reply.
>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>> IIT (BHU),Varanasi,India
>>>>>>>>>>>>>>>>  Mob: +918423542271
>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>> IIT (BHU),Varanasi, India
>>>>>>>>>>>>>>>> Mob: +918423542271
>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Mar 1, 2014 at 9:32 PM, Ershad K <
>>>>>>>>>>>>>>>> ershad92 at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The project is about implementing MARC21[1] data support
>>>>>>>>>>>>>>>>> in Grandham.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Grandham is written in Ruby on rails, the project involves
>>>>>>>>>>>>>>>>> developing MARC21 extensions the platform and integrate it to the current
>>>>>>>>>>>>>>>>> API.  Please drop a mail to discuss at lists.smc.org.in with
>>>>>>>>>>>>>>>>> your preliminary analysis.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1] http://www.loc.gov/marc/bibliographic/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sat, Mar 1, 2014 at 3:43 PM, Deepak kumar <
>>>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello Sir,
>>>>>>>>>>>>>>>>>> I went through your project on grandham and i would like
>>>>>>>>>>>>>>>>>> to contribute to it as my gsoc'14 project.
>>>>>>>>>>>>>>>>>> Please guide me regarding this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>>>> IIT (BHU),Varanasi
>>>>>>>>>>>>>>>>>> Mob: 08423542271
>>>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Ershad K
>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Ershad K
>>>>>>>>>> http://ershadk.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Ershad K
>>>>>>> http://ershadk.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Ershad K
>>>>> http://ershadk.com
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards,
>> Ershad K
>> http://ershadk.com
>>
>
>

-- 
Regards,
Ershad K
http://ershadk.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.smc.org.in/pipermail/student-projects-smc.org.in/attachments/20140311/8655c78d/attachment.html>