[Student-projects] Regarding GSoc 2014

Tue Mar 11 02:55:50 PDT 2014

Hi Deepak,

On Tue, Mar 11, 2014 at 3:08 PM, Deepak kumar <
deepak.kumar.ece11 at itbhu.ac.in> wrote:

> Only the data corresponding to field is of variable length. Control fields
> are of fixed length. Isn't it?
>

Not exactly, I'm not sure how you would parse fields, tags, indicators and
values this way. Have you gone through the MARC21 tutorial I mentioned?

Please see if you can find some library to parse MARC21 and come up with an
example which parses a valid MARC21 record.

>
> On Tue, Mar 11, 2014 at 2:34 PM, Ershad K <ershad92 at gmail.com> wrote:
>
>> It's not fixed length. We need import/export functionality for all valid
>> MARC21 data. How will you parse the different fields?
>>
>>
>> On Tue, Mar 11, 2014 at 2:27 PM, Deepak kumar <
>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>
>>> But every MARC21 record is of the form leader followed by directory of
>>> fields and data corresponding to fields.
>>>  And they are of fixed length. So why we can not use this method?
>>>
>>>
>>> On Tue, Mar 11, 2014 at 2:09 PM, Ershad K <ershad92 at gmail.com> wrote:
>>>
>>>> Deepak,
>>>>
>>>> It doesn't work like that. We cannot parse MARC21 data with fixed
>>>> length strings. Here's the MARC21 tutorial -
>>>> http://www.loc.gov/marc/umb/um01to06.html
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Mar 11, 2014 at 2:04 PM, Deepak kumar <
>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>
>>>>> In view
>>>>> <label for="file">Upload text File</label> <%= file_field_tag "file" %>
>>>>> In controller
>>>>> uploaded_file = params[:file]
>>>>> @file_content = uploaded_file.read
>>>>>
>>>>> In this way @file_content becomes an array of whole record.
>>>>> and by indexing I extracted fields like for leader
>>>>> @leader=@file_content[0..23] since the leader is 24 char. long.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 11, 2014 at 1:59 PM, Ershad K <ershad92 at gmail.com> wrote:
>>>>>
>>>>>> Hi Deepak,
>>>>>>
>>>>>> Can you explain how you parsed the MARC21 file? Also always use
>>>>>> proper classes/controllers, it's good to move such logic out of
>>>>>> UsersController#new.
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 11, 2014 at 1:34 PM, Deepak kumar <
>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>
>>>>>>> the link of code https://github.com/deepakrocks0009/trial
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 11, 2014 at 1:34 PM, Deepak kumar <
>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>
>>>>>>>> Hello Sir
>>>>>>>>
>>>>>>>> I have taken example from http://www.loc.gov/marc/umb/um11to12.html
>>>>>>>> I tried to read MARC record from a text file and output tag,
>>>>>>>> length, start_at and data.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Mar 11, 2014 at 1:57 AM, Ershad K <ershad92 at gmail.com>wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Mar 11, 2014 at 1:53 AM, Deepak kumar <
>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>
>>>>>>>>>> https://github.com/deepakrocks0009/blog  But sir please don't
>>>>>>>>>> judge me on the basis of this code as I was just beginner at that time.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> No worries about that, everybody is beginner at sometime. I just
>>>>>>>>> wanted to see some code in action.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So do I need to create database for this or just reading the
>>>>>>>>>> record from a file?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It's your choice, but for now just parsing a valid MARC21 record
>>>>>>>>> (which you need to find out) is completely fine.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 11, 2014 at 1:48 AM, Ershad K <ershad92 at gmail.com>wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>
>>>>>>>>>>> Please see if you find and parse a MARC21 record. You are free
>>>>>>>>>>> to use any gem.
>>>>>>>>>>>
>>>>>>>>>>> Is deepakkeshri.herokuapp.com open source? Can I see its source
>>>>>>>>>>> please?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 11, 2014 at 1:42 AM, Deepak kumar <
>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> This was my blog app http://deepakkeshri.herokuapp.com/ which
>>>>>>>>>>>> I made when I was just a beginner. After that I learnt a lot of things.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Mar 11, 2014 at 1:34 AM, Deepak kumar <
>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Bhaiya,
>>>>>>>>>>>>>
>>>>>>>>>>>>> You are right. Using STI we will reduce the number of tables.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My github link is github.com/deepakrocks0009. But I dont have
>>>>>>>>>>>>> any big project in my account. The project on which I worked during 2013
>>>>>>>>>>>>> summer was schoolmitra.com and since it was private company
>>>>>>>>>>>>> they closed my bitbucket account after internship.
>>>>>>>>>>>>> Till now I haven't tried to parse MARC records. But you can
>>>>>>>>>>>>> give me task before I submit my proposal.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ya please put this thread to mailing list.
>>>>>>>>>>>>> How many applications do you received for this project?
>>>>>>>>>>>>> Kind Regards
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 11, 2014 at 12:55 AM, Ershad K <ershad92 at gmail.com
>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, but we could make it better using rails's STI. Also
>>>>>>>>>>>>>> have you tried to parse the MARC records? Can you please link to the
>>>>>>>>>>>>>> projects you worked on? Preferably in rails?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can I CC this thread to student-projects at lists.smc.org.in ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> PS: Don't call me sir, call me Ershad :-)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 10, 2014 at 10:54 PM, Deepak kumar <
>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello Sir,
>>>>>>>>>>>>>>> We can have two tables to store MARC 21 data. First table
>>>>>>>>>>>>>>> will contain columns which are field_id, field tag, its value. 2nd table
>>>>>>>>>>>>>>> will contain sub field tag and its value differentiated on the basis of
>>>>>>>>>>>>>>> field_id.
>>>>>>>>>>>>>>> We need to have an extra default row in the table. If the
>>>>>>>>>>>>>>> field doesn't matches existing fields of db then the default row will be
>>>>>>>>>>>>>>> used.
>>>>>>>>>>>>>>> Is this correct?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 10, 2014 at 10:17 PM, Ershad K <
>>>>>>>>>>>>>>> ershad92 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you have any idea about how to store these data in
>>>>>>>>>>>>>>>> database? What happens when we encounter new data fields while importing
>>>>>>>>>>>>>>>> the data? Thoughts?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Mar 10, 2014 at 6:30 PM, Deepak kumar <
>>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You are right sir, the database for the fields is really
>>>>>>>>>>>>>>>>> huge. It has 887 control fields and there sub fields, so total of 6000+
>>>>>>>>>>>>>>>>> field types.
>>>>>>>>>>>>>>>>> It is very difficult to have a look up table for each
>>>>>>>>>>>>>>>>> field. We have to use very good data structure. but there are large number
>>>>>>>>>>>>>>>>> of tools for decoding. We can take help of them.
>>>>>>>>>>>>>>>>> http://www.loc.gov/marc/marctools.html
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sun, Mar 9, 2014 at 3:20 PM, Deepak kumar <
>>>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you sir for your reply.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I am quite good with ruby on rails and its mvc
>>>>>>>>>>>>>>>>>> architecture. The project on which I had worked was
>>>>>>>>>>>>>>>>>> www.schoolmitra.com. It was basically web application
>>>>>>>>>>>>>>>>>> using ruby and its framework RoR.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I have gone through your blog and your weekly update
>>>>>>>>>>>>>>>>>> during GSOC'13 and it is quite impressive.
>>>>>>>>>>>>>>>>>> Some Similar works I also did during my summer
>>>>>>>>>>>>>>>>>> internship. Implementing authentication, post comment and like feature,
>>>>>>>>>>>>>>>>>> student database management, monthly report generation in PDFs form,
>>>>>>>>>>>>>>>>>> defining role through rolify and ability through cancan, bootstrap for css,
>>>>>>>>>>>>>>>>>> SQlite formanaging database, sms alert functionality, javaScript and little
>>>>>>>>>>>>>>>>>> jquery for dynamic pages. But I don't know haml. I used to make my view
>>>>>>>>>>>>>>>>>> page in ruby only. I will learn it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Kind Regards
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Sat, Mar 8, 2014 at 1:31 PM, Ershad K <
>>>>>>>>>>>>>>>>>> ershad92 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sorry, I was busy, wasn't able to send you a reply.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Mar 6, 2014 at 1:34 PM, Deepak kumar <
>>>>>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hello Sir,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> As suggested, I have gone through the documentation of
>>>>>>>>>>>>>>>>>>>> MARC21. What I have understood is, MARC21 is a basically formatting of
>>>>>>>>>>>>>>>>>>>> certain records/data either bibliographic, authority, holdings, or
>>>>>>>>>>>>>>>>>>>> community information. In other words we can say it as encoding and
>>>>>>>>>>>>>>>>>>>> decoding process of records in a fixed format. Today MARC21 is most
>>>>>>>>>>>>>>>>>>>> predominant and widely used. So in order to import data from a MARC21
>>>>>>>>>>>>>>>>>>>> record we have to read the record character by character and perform
>>>>>>>>>>>>>>>>>>>> decoding operation. The project says we have to import and export data in
>>>>>>>>>>>>>>>>>>>> MARC21 records. The only tough thing about this project is to know the
>>>>>>>>>>>>>>>>>>>> MARC21 format.
>>>>>>>>>>>>>>>>>>>> So my steps will be
>>>>>>>>>>>>>>>>>>>> 1. Reading the MARC21 record file as a string or
>>>>>>>>>>>>>>>>>>>> reading fixed number of character from record each time instead of reading
>>>>>>>>>>>>>>>>>>>> whole record as single string.
>>>>>>>>>>>>>>>>>>>> 2. Assigning these fixed length strings to fixed
>>>>>>>>>>>>>>>>>>>> variables.(Tag,length,start position,end position)
>>>>>>>>>>>>>>>>>>>> 3. With the help of these tags reading the rest of file.
>>>>>>>>>>>>>>>>>>>> 4. Lastly formatting these pieces of data in fixed
>>>>>>>>>>>>>>>>>>>> format for end users with the help of inbuilt database.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> There's some problem here. First, we cannot really read
>>>>>>>>>>>>>>>>>>> MARC21 records by reading with a fixed number of characters. Last time I
>>>>>>>>>>>>>>>>>>> checked, it has around 3000+ different fields with different type of data.
>>>>>>>>>>>>>>>>>>> Each file has many subfield marked with leader tags or something. We cannot
>>>>>>>>>>>>>>>>>>> proceed with this project until we have a clear understanding of how MARC21
>>>>>>>>>>>>>>>>>>> works.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In addition to that, the scope of this project is not
>>>>>>>>>>>>>>>>>>> merely parsing the MARC21 data. We have to make necessary changes in
>>>>>>>>>>>>>>>>>>> Grandham application to add import/export feature. That includes changes in
>>>>>>>>>>>>>>>>>>> the current DB schema. How good are you with Ruby on Rails?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> This will be basic idea of my work. Please correct me
>>>>>>>>>>>>>>>>>>>> wherever I am wrong also if I am missing any point.
>>>>>>>>>>>>>>>>>>>> Please tell when you are available on IRC . I have
>>>>>>>>>>>>>>>>>>>> posted this mail to mailing list but didn't got any reply.
>>>>>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>>>>>> IIT (BHU),Varanasi,India
>>>>>>>>>>>>>>>>>>>>  Mob: +918423542271
>>>>>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>>>>>> IIT (BHU),Varanasi, India
>>>>>>>>>>>>>>>>>>>> Mob: +918423542271
>>>>>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Sat, Mar 1, 2014 at 9:32 PM, Ershad K <
>>>>>>>>>>>>>>>>>>>> ershad92 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Deepak,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> The project is about implementing MARC21[1] data
>>>>>>>>>>>>>>>>>>>>> support in Grandham.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Grandham is written in Ruby on rails, the project
>>>>>>>>>>>>>>>>>>>>> involves developing MARC21 extensions the platform and integrate it to the
>>>>>>>>>>>>>>>>>>>>> current API.  Please drop a mail to
>>>>>>>>>>>>>>>>>>>>> discuss at lists.smc.org.in with your preliminary
>>>>>>>>>>>>>>>>>>>>> analysis.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> [1] http://www.loc.gov/marc/bibliographic/
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 1, 2014 at 3:43 PM, Deepak kumar <
>>>>>>>>>>>>>>>>>>>>> deepak.kumar.ece11 at itbhu.ac.in> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hello Sir,
>>>>>>>>>>>>>>>>>>>>>> I went through your project on grandham and i would
>>>>>>>>>>>>>>>>>>>>>> like to contribute to it as my gsoc'14 project.
>>>>>>>>>>>>>>>>>>>>>> Please guide me regarding this.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Deepak Kumar
>>>>>>>>>>>>>>>>>>>>>> Electronics Engineering
>>>>>>>>>>>>>>>>>>>>>> B.Tech III Year
>>>>>>>>>>>>>>>>>>>>>> IIT (BHU),Varanasi
>>>>>>>>>>>>>>>>>>>>>> Mob: 08423542271
>>>>>>>>>>>>>>>>>>>>>> http://about.me/deepak.keshri/<http://about.me/deepak.keshri/#>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Ershad K
>>>>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ershad K
>>>>>>>>>>> http://ershadk.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Ershad K
>>>>>>>>> http://ershadk.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Ershad K
>>>>>> http://ershadk.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Ershad K
>>>> http://ershadk.com
>>>>
>>>
>>>
>>
>>
>> --
>> Regards,
>> Ershad K
>> http://ershadk.com
>>
>
>

-- 
Regards,
Ershad K
http://ershadk.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.smc.org.in/pipermail/student-projects-smc.org.in/attachments/20140311/1de4614a/attachment.html>